INTOSAI Privatisation Working Group PWG

1
INTOSAI Privatisation Working Group (PWG)

• Technical Case Study
• Series 3 Economic Regulation

1 Effectiveness of Regulatory Frameworks
2
Summary

• Regulation is seen as an important factor in
  economic development and social welfare.
• But regulation can impose direct and hidden
  costs on economic activity and can inhibit risk
  taking.
• - Ensuring that regulation is well designed and
  implemented is a growing challenge for
  Government.
• Is there a role for the SAI in improving the
  effectiveness of regulatory frameworks?
• It is important to remember that whilst
  government and regulators should be accountable
  for the efficiency and effectiveness of their
  activities, regulators should remain rigorously
  independent in the decisions they take. As
  such, the SAI must ensure it does not seek to
  control the design or operation of regulatory
  frameworks.
• Auditors must therefore adopt the role of
  evaluating and scrutinising the effectiveness of
  regulation / regulatory frameworks, with the key
  considerations being whether regulatory
  frameworks are cost- effective, and whether they
  are delivering the intended outcomes.
• The unique selling point of the SAI is its
  independence. This additionally gives it a major
  role in the objective assessment of government
  performance in reducing burdens. In all these
  areas, the SAI can pass on knowledge of best
  practice, and ensure lessons previously learnt in
  setting up and operating regulatory frameworks
  can benefit all, particularly through the
  following roles
• Assessing the process by which regulation is
  designed, through examination of the
  effectiveness of ex- ante tools such as Impact
  Assessments, and ex-post tools which examine the
  costs or burdens imposed by regulation and
• Assessing the effectiveness and efficiency of
  regulators as institutions, including their
  structure, their risk focus, and their
  enforcement approach.

3
1. Regulation as an important factor in
economic developmentRegulation and privatisation

• The growth in regulation
• The role of any regulation is to ensure that an
  entity complies with a certain area of the law
  the absence of such laws can lead to damaging
  consequences for consumers, and society more
  widely.
• There are many risks in a competitive market
  which can lead to market failure, providing
  opportunities for powerful entities to exploit
  individuals or whole sections of society. These
  risks have been a driving force in the growing
  use of statutory rules as the preferred tool of
  choice for governments to modify or constrain
  certain behaviours, or the perceived risk of
  these behaviours.
• As a society, we have increased expectations that
  regulatory frameworks can and will protect
  consumers, businesses, workers and the
  environment. This has led to a growth in
  regulation, which is now coupled with an
  increasing need to keep our businesses efficient
  and flexible to face new competitive challenges.
• Motivation for the creation of regulators
• The growth of privatisation in recent years
  reflects a desire by governments to introduce the
  benefits of a competitive market to previously
  state-run industries, in order to provide more
  value for money for consumers. A major
  consequence of these privatisation policies is
  that many countries have moved from being
  producing states to regulating states, with
  large and significant industries now operating
  subject to market forces rather than state
  control.
• The transition to a fully competitive market can
  be complex however, and separate regulatory
  institutions have often been created to oversee
  this process, whilst protecting the interests of
  consumers. These bodies are often highly
  specialised and industry-specific.
• Once the transition is completed, privatisation
  also introduces the risks of a competitive market
  to the former state-run industries. These are
  often major industries such as energy, transport
  and banking, and the consequences of a failure to
  deliver an adequate or economic level of service
  can have potentially damaging consequences for
  consumers in particular, but also businesses,
  workers, and the environment. To mitigate these
  risks, many governments have used the separate
  regulatory bodies to prevent market abuse or
  market failure occurring.

4
1. Regulation as an important factor in
economic developmentRisks in a competitive
market

• Risk of market abuse by dominant companies
• Where one company is dominant within a specific
  market, this may provide it with more incentive
  to charge excessive prices or provide poor
  service. One potential consequence of
  privatisation is that a public monopoly is
  replaced by a private one. In such circumstances,
  the creation of an independent regulator is a
  means of trying to prevent market abuse.
• In the UK, for example, the privatisation of
  water companies as a series of regional
  monopolies was accompanied by the creation of the
  Water Services Regulation Authority (Ofwat) as
  water regulator, with responsibility for
  controlling the price and quality of service
  offered by private sector water companies.
• Risk of wider market failure
• While markets may be efficient in matching
  resources to demand, they may ignore or even
  create wider external problems, such as the
  pollution problems associated with energy
  production. The typical policy response to such
  market failures is to create regulations designed
  to prevent and prohibit the market failure. In a
  privatised industry, the independent regulator
  may well be charged with this role.
• An example of this is the Office of Gas and
  Electricity Markets (Ofgem) in the UK, which in
  addition to promoting competition in the UK
  energy industry, is also charged with
  administering the Climate Change Levy on energy
  providers.

5
1. Regulation as an important factor in
economic developmentRisks in a competitive
market

• Risk of uncompetitive behaviour
• Even competitive markets can sometimes relapse
  into anticompetitive behaviour. This can often
  manifest in the form of cartels, or other
  restrictions that harm consumers, denying them
  the benefits of a choice between competing
  suppliers. In such circumstances, general market
  regulation or industry-specific regulators may be
  a tool enabling government intervention.
• Example OFT?
• Risk of closed markets
• Markets thrive on clear, open, transparent rules,
  where all players have equal access to the
  market. It is often the case that some businesses
  have better access to a market, possibly due to
  geographical location, causing possible harm to
  more restricted businesses and consumers.
  Governments may seek to regulate to enhance the
  operation of the market in such instances, as is
  the motivation behind much EU legislation.
• In a privatisation context, a similar situation
  arises when the delivery network for the former
  state-run industry is controlled by one of the
  players in the newly created market, allowing it
  a considerable advantage over its new
  competitors. A key function of a regulator in
  this situation will be to ensure fair access to
  the network to support a competitive market.
• Example? BT / Royal Mail

6
2. The potential problems of regulationCosts
of regulation

• Regulation as a convenient tool for government
• Formal regulation through statutory rules can
  typically be seen by governments as the quickest,
  easiest and most publicly visible tool for
  delivering the required effect, especially in
  times of crisis. However, outdated or poorly
  designed regulatory frameworks can impede
  innovation and establish barriers to entry,
  creating unnecessary barriers to trade,
  investment and economic efficiency.
• Direct and hidden costs of regulation
• The costs to business of complying with
  regulations are generally divided into three
  types
• Administrative costs (the cost of providing
  evidence to Government to demonstrate
  compliance)
• Financial costs (such as paying tax or licence
  fees) and
• Policy costs (the cost of complying with the
  actual policy objective of any given regulation).
• Administrative and financial costs may both be
  classified as direct costs to business. They
  include time and money spent on formalities and
  paperwork necessary to comply with Government
  imposed regulations. This covers the information
  gathering process, reporting and notification
  requirements, applying for any relevant permits,
  and showing inspectors around a site.
• Policy costs, in addition to direct costs, also
  impose indirect costs on business. Indirect costs
  arise when compliance with regulatory frameworks
  reduces the productivity and innovation of
  enterprises. Some regulation can be very
  prescriptive, stipulating processes to be
  followed in addition to the outcomes to be
  reached. This is sometimes referred to as
  command and control regulation. The
  alternative, principles based regulation, is
  argued to reduce burdens by only specifying
  outcomes and therefore allowing more scope for
  private sector innovation in delivery.
• As policy costs are more difficult to determine
  than direct costs, it is not surprising the OECD
  has stated that regulatory costs are the least
  controlled and least accountable amongst
  government costs.

7
3. The role of the SAI in improving the design
of regulatory frameworksAssessing the design
process for regulatory frameworks

• Ex-ante tools Impact Assessment
• Impact Assessment (IA) is a tool widely used
  across governments which aims to ensure that
  regulatory proposals are subject to a
  transparent, publicly accountable and rigorous
  analysis to determine if they are a proportional
  means of meeting regulatory objectives. Used
  effectively, they help ensure the burden of
  regulation is not unnecessarily increased by
  answering the questions
• Should a new regulation be introduced?
• Should an existing regulation be abolished?
• Are there better ways of achieving the objective
  rather than regulation?
• Do the benefits justify the costs?
• What is the most cost-effective option?
• The IA is designed to enhance policy making by
  identifying the option which best
• Focuses on the problem
• Responds proportionally to the risk
• Is seen as user friendly
• Can be fairly administered
• Has coverage of the relevant population
• Allows sanctions which are appropriate
• Provides social benefits of compliance which
  exceed social costs of compliance and
• Is open to review and evaluation.

8
3. The role of the SAI in improving the design
of regulatory frameworksAssessing the design
process for regulatory frameworks

• Ex-ante tools Impact Assessment
• It is not the role of an SAI to form an opinion
  on the outcome of an IA, as whether the best
  option has been chosen is a policy matter to be
  decided by government, based on the information
  and evidence collected from experts in the
  relevant area.
• There is however an important role for the SAI to
  play in examining the use of IAs across
  government, including regulatory bodies, and the
  processes that have been undertaken throughout
  the development of policy options. The SAI can
  examine if adequate mechanisms are in place to
  encourage high quality IAs, with the principles
  of IA being applied throughout the policy
  process. The SAI can also examine if the evidence
  base used in IAs is timely and robust.
• By adopting a suitable evaluation framework, the
  SAI can assess whether the IA process is
  effectively challenging the need for new
  regulation. Key questions in such a framework
  would include
• Was the scope and purpose of IAs clearly defined?
• Was consultation effective?
• Were costs and benefits assessed thoroughly and
  realistically?
• Did the IA realistically assess compliance?
• Will the regulation be effectively implemented,
  monitored and evaluated?
• Did the IA consider the impact of the regulation
  on competition?
• E.g. Work by the UK NAO highlighted how one
  department was using a phased roll-out of
  regulations, enabling cost estimates included in
  IAs to be tested before a full roll-out took
  place.

9
3. The role of the SAI in improving the design
of regulatory frameworksAssessing the design
process for regulatory frameworks

• Ex-post tools costs of regulation
• Better regulation can also be achieved by a
  consideration of regulatory issues throughout the
  complete policy cycle. Post implementation,
  evaluation of the actual additional costs being
  incurred will allow analysis of the robustness of
  figures on which the decision making process was
  based, to improve the quality of future IAs.
• This analysis of the costs of regulation will be
  initially restricted to the direct administrative
  costs, as the more indirect policy costs take
  longer to emerge. This area of administrative
  burdens is one in which, since 2003, countries
  such as the Netherlands, Denmark and Sweden have
  begun to develop the Standard Cost Model (SCM) in
  an attempt to establish a standard measurement
  tool. This has the dual role of allowing targets
  to be set for burden reduction, as well as
  facilitating checks on the performance of
  regulation against initial assumptions.
• Again, whilst such analysis is the task of
  government, there is an important role here for
  the SAI in assessing the degree to which this
  tool is being effectively deployed across
  government, including regulatory bodies. This
  role thus sits neatly in conjunction with the
  assessment of IAs as previously described.

10
3. The role of the SAI in improving the design
of regulatory frameworksAssessing the design
process for regulatory frameworks

• Ex-post tools post implementation review
• Further to the analysis of the administrative
  costs imposed by regulation, the policy cycle is
  completed by a full post implementation review
  involving the full range of estimates and
  assumptions on which the regulatory framework was
  based. This requires the regulator to consult
  with the relevant stakeholders, and allows more
  consideration of the more indirect policy costs
  as well as wider industry issues.
• Though often neglected, this process is key to
  ascertaining if the stated objectives of
  regulatory frameworks are being met, if there is
  scope to further reduce the regulatory burden,
  and if specific regulation is proving effective
  or should perhaps even be removed. Plans for the
  review should be included as part of the IA
  process, with the feedback and lessons learned
  from the review being used to improve the IA
  process in the future.
• The best way for the SAI to contribute to this
  process is again to assess how well this tool is
  being used by regulators. It can then help to
  ensure that lessons are learnt from situations
  where such processes have been viewed as
  successful, enabling best practice to be drawn
  upon from across government and any flaws in the
  process to be avoided.
• E.g. Work by the UK NAO highlighted one
  department with a coordinated approach to post
  implementation reviews, whereby the learning
  points identified were fed back to not only
  policy teams, but into improved IA guidance and
  training.

11
3. The role of the SAI in improving the design
of regulatory frameworksPassing on knowledge of
best practice

• Contribute to a central government regulation
  oversight body
• Following OECD recommendations, many countries
  are establishing central government oversight
  bodies for the area of regulation. This is as
  part of the drive to create more authoritative
  and transparent regulatory structures. In
  addition to addressing regulation across
  government generally, such bodies also tend to
  have the following responsibilities
• holding regulators to account
• introducing reforms with a view to reducing
  administrative burdens and
• providing advice on risk assessment, both in
  theory and in practice.
• The SAI, whilst maintaining clear independence
  from bodies that are within the structures of
  government and avoiding interference in policy
  formation, is well placed to make strong
  contributions to such oversight bodies through
  its knowledge of best practice in the area of
  regulation. This can help further improve
  regulatory frameworks.
• E.g. In the UK the findings from NAO reports have
  provided the Better Regulation Executive with a
  strong evidence base on the effectiveness of the
  regulatory approach, thus contributing to the
  development of guidelines for the IA process
  which are used across government.

12
3. The role of the SAI in improving the design
of regulatory frameworksPassing on knowledge of
best practice

• Assessing progress on regulation reduction
  programmes
• As well as introducing programmes to improve the
  quality of regulatory frameworks, some
  governments are also undergoing regulation
  simplification programmes. The experience of the
  Dutch government has shown however that whilst
  reduction targets have been achieved, business is
  sceptical as they have not seen a proportionate
  impact on their regulatory administrative duties
  to support this.
• Where such simplification programmes are
  undertaken, there is again an important role for
  the SAI in auditing the robustness of the
  governments claimed reductions in administrative
  costs. The SAI should be careful not to become
  involved at the start of such programmes, in
  setting up the tools to be used for measuring
  impacts claimed as reduced burdens. Rather, it
  has the important function of reviewing and
  helping to improve the governments approach to
  measuring impacts and the tools being used for
  this purpose.
• This sort of work by the SAI can act to motivate
  the government to ensure they meet regulatory
  reduction targets, and help the government to
  identify the key drivers in this process, both of
  which are key elements in ensuring that more
  efficient regulatory frameworks are in place.

13
4. The role of the SAI in improving the
effectiveness of regulatory frameworksAssessing
the effectiveness of regulators as institutions

• Regulatory independence
• Economic regulation has the key role of ensuring
  that regulated businesses supply essential
  services at a fair price to the public. Thus the
  consumer needs to be confident that the
  regulatory body is capable of defending their
  interests in the face of powerful suppliers,
  whilst suppliers need to be confident that the
  regulatory body will not prevent them making a
  fair return for a fair level of service.
• To achieve this delicate balance, many regulators
  have been established as arms length bodies from
  government. This independence allows a long term
  perspective to be taken when regulating,
  beneficial for achieving appropriate investment
  by suppliers in an industry. Independence also
  allows more objectivity and openness as decisions
  can be taken impartially in the public interest,
  and free from political pressure or pressure from
  regulated suppliers.
• Regulated industries frequently tend to be
  essential services however, so in the case of a
  failing regulator governments will therefore feel
  obliged to become politically involved. In some
  instances, this obligation can slide into
  political interference with a consequential
  impact on the effectiveness of the regulatory
  framework. If this occurs, the SAI is well placed
  to examine any additional costs which arise from
  a negative impact on the effectiveness of the
  regulatory framework, and highlight the reasons
  for this.

14
4. The role of the SAI in improving the
effectiveness of regulatory frameworksAssessing
the effectiveness of regulators as institutions

• Regulatory independence
• Another important factor in the structure of the
  regulatory environment is the form which consumer
  representation takes. Consumer panels may be
  established independently of regulators, with the
  role of pressuring and influencing their
  decisions (E.g. Energywatch and Postwatch).
  Alternatively, they may be situated within the
  regulator itself so that this perspective can be
  fully incorporated before regulations are
  finalised (E.g. Ofcom).
• The SAI is able to examine the structure of
  consumer representation to establish whether this
  offers value for money. There may be duplication
  of effort between consumer bodies, which could
  better be merged. Alternatively, consumer bodies
  may merely be replicating functions already
  performed by the regulator, hampering effective
  regulation through inappropriate challenges to
  regulation, or perhaps would be better placed
  inside the regulator.
• E.g. Postcomm, where most decisions taken are
  either appealed by Royal Mail, or if not, then by
  Postwatch instead.
• The above structural factors can significantly
  alter the effectiveness of a regulators
  operations. Consequently, due to the size of GDP
  often represented by the regulated industries,
  this can have a significant impact on consumers.
  It therefore falls within the remit of the SAI to
  assess whether the regulatory structure in place
  is the most efficient and effective. The SAI may
  seek to do this by examining factors including
  how regulators are funded and organised in
  addition to the process by which they establish
  policy.

15
4. The role of the SAI in improving the
effectiveness of regulatory frameworksAssessing
the effectiveness of regulators as institutions

• Regulatory accountability assessing the
  performance of the regulator
• As independent bodies accountability is a key
  issue for regulators, with four distinct strands
  being relevant, i.e. financial, policy, judgement
  and procedural. Regulators are already
  accountable for their financial performance
  through the SAI, and they are accountable for
  policy to parent departments. They are also
  generally accountable for the judgements they
  make through appeals procedures. In many
  countries however they are currently not
  accountable for the way in which they carry out
  their work. This reduces incentives for
  regulators to join up work, and promotes
  inconsistency.
• A fully accountable regulator will increase the
  confidence of the industry and other stakeholders
  in its operation, and thus improve the
  effectiveness of the regulatory framework it is
  overseeing. The SAI again has a central role in
  establishing the accountability of the regulator,
  by examining the way it operates procedurally.
• Regulators objectives should be clear, and
  mapped to a set of appropriate key performance
  indicators (KPIs) which are specific, measurable,
  achievable, relevant and time-bound (SMART),
  leading to more focussed efforts and stronger
  outcomes. When the regulators report on these,
  they should be held accountable for them. The SAI
  is well placed to perform this role, and such
  involvement will provide transparency to the
  regulation activity and give weight to the work
  of the regulator.
• E.g.??

16
4. The role of the SAI in improving the
effectiveness of regulatory frameworksAssessing
the effectiveness of regulators as institutions

• Quality of Staff and Management
• Economic regulation is a technically complex role
  requiring a detailed knowledge of economic issues
  such as competition and the operation of markets,
  along with a detailed understanding of consumer
  interests and needs. On top of this, each
  regulator will need to have top quality expertise
  in the specific sector they are operating in.
  These qualities should be reflected in all staff
  involved in the regulators core work, from
  management downwards, thus requiring it to have
  appropriate recruitment policies. In addition,
  access to the relevant external expertise will
  also be required without comprising independence.
• Furthermore, a careful balance needs to be struck
  between being too detached from the suppliers in
  their industry, so that the staff are uninformed,
  and being too close to the industry. If the
  regulator is too keen to co-operate with the
  industry this could harm consumers both directly,
  and also indirectly by stifling new entrants to
  the market and reducing competition as a result.
• The SAI is in a position to examine the degree of
  technical competence within a regulator, and the
  impact this has on the effectiveness of their
  regulation. It can also look at the mix of
  in-house and external expertise a regulator is
  using and assess whether this is the most
  efficient use of resources.
• Aside from the sector-specific expertise that
  regulators require, there may also be scope for
  the SAI to look for efficiencies in the more
  general, and therefore shared, areas of
  regulatory expertise which feature across the
  whole of a governments regulatory structure.
• E.g. ??

17
4. The role of the SAI in improving the
effectiveness of regulatory frameworksAssessing
the effectiveness of regulators as institutions

• Rule based vs. principle based regulation
• In recently privatised markets governed by a
  regulator, there is a temptation for the
  regulator to be prescriptive in the way it
  regulates due to its detailed knowledge of the
  industry. This would mean that in addition to
  stipulating outcomes, such as service levels and
  prices, it also largely stipulates the methods by
  which the companies operate. In some instances,
  such an approach is necessary as it is required
  to address market failures and ensure that vital
  interests, e.g. environmental, or health and
  safety standards, are protected.
• In recent years however a counter-view has
  developed claiming that such an approach stifles
  private sector innovation and therefore reduces
  the benefits of privatisation which could
  otherwise be passed onto the consumer.
• Another way in which regulators can have negative
  impacts on supplier efficiency is through poorly
  designed price control regimes. This can damage
  the functioning of the market when some of the
  suppliers costs are out of their control, e.g.
  in energy markets, or when the ability of
  suppliers to invest in infrastructure or raise
  finance is impacted, as has been claimed of the
  RPI-X regime, e.g. Ofwat.
• Within its remit, the SAI can examine the
  effectiveness with which a regulator is
  discharging its duties. This can involve an
  examination of the above issues in order to
  establish if the industry is functioning
  effectively for the consumer. Examples, domestic
  or otherwise, where principle based regulation
  has allowed scope for innovation which improves
  supplier performance may be highlighted as best
  practice, and then used to demonstrate any
  problems detected from an over-reliance on using
  rule based regulation.
• Also, further to examining the evidence used to
  establish a price control regime, the SAI can
  look at the effect of a price control regime on
  an industry. This can involve analysing the
  relative costs of investing in infrastructure at
  different times, and concluding whether the
  effect of any price control regime in place is to
  inhibit optimal investment strategies from being
  followed.
• E.g. ??

18
4. The role of the SAI in improving the
effectiveness of regulatory frameworksAssessing
the effectiveness of regulators as institutions

• Risk assessment contributing to an effective
  and efficient inspection regime
• One area that can consume much of a regulators
  resource is inspections. In the UK, a recent
  review has considered the scope for reducing
  administrative burdens by promoting more
  efficient approaches to regulatory inspection and
  enforcement, without compromising regulatory
  standards or outcomes. This report identified
  risk assessment as fundamental to effectiveness
  and an essential means of directing regulatory
  resources where they can have the maximum impact
  on outcomes.
• Risk assessment should be comprehensive, and
  inform all aspects of the regulatory lifecycle
  from the selection and development of appropriate
  regulatory and policy instruments through to the
  regulators work including data collection,
  inspection and prosecution.
• The SAI can aid in risk assessment through
  passing on best practice it has experienced in
  auditing other regulators and its statutory
  clients, many of whom are required to carry out
  comprehensive inspection programmes encompassing
  elements of risk and random selection criteria.
  They can also analyse any methodology which the
  regulator has introduced for identifying those
  areas which pose the greatest risk, and therefore
  would be subject to the highest level of
  inspection activity, to ensure this reflects the
  true structure of the industry.
• E.g. In Brazil the SAI was involved in the
  universalisation of the telecommunications
  service by helping to recommend an inspection
  strategy taking into account efficiency,
  effectiveness and economicity criteria.

19
4. The role of the SAI in improving the
effectiveness of regulatory frameworksAssessing
the effectiveness of regulators as institutions

• Use of enforcement powers
• There are a range of regulatory breaches which
  may cause a regulator to impose penalties on a
  supplier in a regulated industry, including
• Failing to report appropriately / reporting
  incorrect information
• Poor service quality
• Failing to adhere to price control regimes
• Following anti-competitive practices and
• Failing to provide the necessary prescribed
  investment.
• In a competitive market such breaches may come to
  light through a range of channels, including the
  inspection regime or notification by interested
  stakeholders, e.g. consumer groups or rival
  suppliers. Regulators however remain dependant on
  suppliers for the provision of accurate
  information, yet suppliers have an incentive to
  misreport if it will lead to a competitive
  advantage.
• To be effective in such a situation, a regulatory
  framework needs to be enforced appropriately. In
  addition to an effective inspection regime, this
  requires regulators to use their sanctions
  (ranging from conditions attached to licences, to
  financial penalties, or even revoked licences) in
  a manner that will most benefit the functioning
  of the market and therefore ultimately the
  consumer.
• The SAI can assist by considering what measures
  the regulator can take to reduce its information
  disadvantage. This will involve reviewing the
  regulators information needs and the format in
  which they collect it, ensuring it allows
  comparison across the industry. It can also
  review the incentives used to encourage the
  provision of accurate information.
• Furthermore, the SAI can examine whether the
  regulatory regime contains and utilises penalties
  which are proportionate to the gains of
  non-compliance. In particular, these can be
  assessed as to the extent to which they improve
  the functioning of the market and benefit the
  consumer.