Client Information Session

1
Client Information Session

• Performance Management Systems Audit Update
• 28 May 2007

2
Agenda

• Welcome and Update by Glenn Poole Auditor-General
• Management of Funding to NGOs
• Capital Works
• Whole-of-Government Risk Management
• Looking ahead

3
Welcome and Update

• Glenn Poole
• Auditor-General

4
Management of Funding to NGOs
Auditor-General Report No. 2 for 2007

• Irene Sitton
• Audit Manager
• Queensland Audit Office

5
Overview

• Audit Process
• Topic Selection
• Research Process
• Planning
• Auditees Perspective
• Sector Wide Findings

6
PMSA

• s.80 FAA Act
• independently examine whether performance
  management systems enable management to assess
  achievement of objectives economically,
  efficiently and effectively.

7
Criteria for Topic Selection

• Overall Estimated Audit Impact
• Financial Materiality
• Risk to Good Management
• Significance of the Topic
• Visibility / Public Interest
• Previous Audit Coverage
• Auditability

8
Phases of a PMSA
9
PMSA Decision Tree
Topic Identification
Audit Scoping Exercise
Yes
No
PLANNING
Retain on audit watch list
IMPLEMENTATION
REPORTING
10
Understanding the topic

• Environmental Scanning
• Potential Auditee Consultation
• Other Stakeholder Consultation

11
Environmental Scanning

• Legislative and Policy Framework
• QAO and other Audit Reports
• Literature Review
• Government Reports
• Benchmarking to other Jurisdictions

12
Funding to NGOs

• Legislative and Policy Framework
• FAA, FMS, Guidelines for Grant Administration.
  Queensland Treasury 1997.
• QAO and other Audit Reports
• Grants Management Enhancing Performance
  Accountability Provisions Could Lead to Better
  Results USA GAO 2006.
• Grants Management Grantees Concerns with
  Efforts to Streamline and Simplify Processes USA
  GAO 2006.
• Working with the Third Sector UK NAO June 2005.
• Literature Review
• Accountability of Australian Nonprofit
  Organisations Reporting Dilemmas Flack, T and
  Ryan, C. 2003
• Third Sector The contribution of non-profit and
  cooperative enterprise in Australia Lyons, Mark.
  2001.
• The Grantmaking Tango Issues for Funders. The
  Baring Foundation. Unwin, J. 2004. London.
• Government Reports
• Strategic directions Investing in Queenslands
  Community, services. Consultation Paper.
  Community Services Strategy Department of Premier
  and Cabinet. 1999.
• Realising the vision. Governance for the Smart
  State. Office of Public Service Merit and Equity.
  2003
• Seamless government Improving outcomes for
  Queenslanders, now and in the future. OPSME
  2004.
• Strengthening Non-Government Organisations
  Report. Queensland Government. 2005.
• Community Services in Queensland An
  Environmental Scan. A Community Services Strategy
  Research Paper. Community Services Strategy
  Secretariat. Social Policy Department of the
  Premier and Cabinet. 2000.

13
Funding to NGOs

• give (or donate) money to worthy causes such as
  scientific and charitable foundations
• shop for (or procure) specialist social services,
  such as disability, counselling and health
  services for individuals or families in need in
  our community
• invest through providing infrastructure or
  training to build and strengthen our local
  communities.

14
PMSA Planning
REPORT
Audit Strategy
The audit costs should be reasonable in relation
to the benefits likely to result from the audit
strategy
Planning means developing a general strategy and
a detailed approach for the expected nature,
timing and extent of the audit.
Assess Staff Skills
Sources of Evidence
Set Milestones
Audit Criteria
Audit Objective
Potential Issues
Audit Scope
Set Budget
Given Audit Objective, Scope, Criteria and
Sources of Evidence are known develop Audit Work
Plan or Program
Planning Outputs - ? Audit Work Plan / Program ?
Audit Strategy ? Audit Budget ? Letter to Client
to Advice of Commencement of audit ?
Client Information Package
The work plan contains the audit procedures to be
done to implement the audit
15
Audit Objective

• Do agencies have suitable frameworks and
  appropriate systems in place to plan, monitor and
  evaluate the provision of community services
  through funding non-government organisations
  (NGOs)?
• How do these systems shape and sustain their
  relationships with NGOs and their extensive
  network of NGO service providers?

16
Audit scope

• Matters to be examined
• Time period covered by the audit
• Entity or part thereof subject to audit
• Source AUS806/808.

17
Audit Scope What?

• Two lines of audit enquiry
• Focus on the performance management systems in
  place that sustain relationships with the NGO
  sector in the context of the Strengthening NGO
  and Partnering Policy Frameworks.
• Focus on the systems in place to plan, resource,
  monitor and evaluate funded programs delivered by
  NGOs.

18
Audit Scope When?

• Field visits JanuaryFebruary 2007
• Financial year end 2006

19
Audit Scope Who?

• Department of Child Safety (DChS)
• Department of Communities (DoC)
• Disability Services Queensland (DSQ)

20
Auditee Profile
DChs
DSQ
DoC
Annual spent on grants and subsidies
of total expenditure
21
Auditee Profile

• DoC was rolling out their needs based planning
  and resource allocation framework, planning for
  new quality systems, IT grants management systems
  and legislative change. The focus was strategic.
• DSQ was implementing new quality, new grants
  management IT systems and new funding agreements.
  The focus was operational.
• DChS was finalising the CMC BluePrint. The
  focus was embedding organisational growth
  development.

22
Key Findings

• Each audited department demonstrated particular
  strengths in their approach to funding the NGO
  sector
• DoCs approach to needs based planning and
  resource allocation and its first stage
  development of information systems has
  established a good foundation for planning.
• DSQ in its implementation of quality systems has
  demonstrated strengths in systems implementation
  and in managing change with the NGO sector.
• DChS through its annual reporting confirms the
  benefits of transparency in public disclosures.

23
An Auditees Perspective

• Anne Reddell
• Executive Director Community Funding
  Sector Development
• Department of Communities

24
An Auditees Perspective

• Ruth Brown
• Director Grants Management Branch
• Disability Services Queensland

25
Sector-Wide Observations

• Governance
• Systems
• Reporting

26
Governance

• Refining legislation and policy frameworks
• Clarifying roles and responsibilities
• Moving to common standards
• Aligning grant approval delegations to risk

27
Systems

• Improving information systems
• Collecting and using the right information
• Evaluating results
• Reducing red tape

28
Reporting

• Increasing public disclosures

29
Recommendations

• Opportunities for sector wide improvements
  included
• The development of supporting cross functional IT
  systems to aid in the collection and analysis of
  data.
• The development of a whole-of-government policy
  which articulates the interaction framework to be
  applied by individual departments and facilitates
  a coordinated approach to funding and
  accountability requirements across departments.
• The development of an overarching policy which
  identifies different government/NGO interactions,
  such as giving, shopping and investing combined
  with the use of one funding terminology of
  grants being discontinued.

30
Balancing Conformance with Performance
31
Capital Works
Auditor-General Report No. 5 for 2006

• Richard Heinritz
• Audit Principal
• Queensland Audit Office

32
Audit Objective

• To determine whether budget sector agencies had
  suitable frameworks and systems in place to
  support the effective management of their capital
  works buildings program. In particular, the audit
  assessed the level of adherence to the Capital
  Works Management Framework (Framework)
  established by Government to assure cost
  effective management of capital works activities.

33
Audit Sample

• 4 Departments Corrective Services, Education,
  Health and Housing. The audit focused on the
  project planning, project delivery, project
  reviews and project governance of capital works
  management and reviewed a sample of 24 building
  projects ranging in cost from 0.36m to 142.5m

34
Capital Works Management Framework

• The Audit assessed compliance with Governments
  Capital Works Management Framework to assure
  cost-effective management of capital works
  activities.

35
Audit Conclusion

• 4 Departments had in place a range of systems at
  varying levels of maturity to manage their
  capital works program. Across the 4 agencies
  audited, compliance with the framework ranged
  from substantial to limited.

36
Audit Conclusion

• Capital works projects being commenced prior to
  clearly establishing whether the project is the
  most effective service delivery solution.
• Inaccurate project cost estimates, with the final
  expenditure being up to 120 per cent higher due
  to not determining the full scope of the project
  prior to commencement and
• Lost opportunity to improve future performance
  due to not undertaking post construction reviews .

37
Key Audit Findings
38
Project Planning

• Align Capital works to community needs
• Options and Project Analysis
• Cost Estimates lack of suitable estimation
  processes

39
Project Delivery

• Agreements with service providers not always
  formalised
• Project Reviews

40
Project Governance

• Records Management

41
Key Recommendations

• Project Planning
• Align capital works to community needs
• Public sector entities define and document how
  individual projects align with the current and
  future service delivery needs of the community
  prior to commencement of new capital works.

42
Key Recommendations

• Options and project analysis
• Public sector entities analyse, consider and
  document a range of project options to ensure
  that the preferred project solution clearly
  demonstrates the most advantageous outcome for
  the community.

43
Key Recommendations

• Cost estimates
• Public sector entities adopt cost estimation
  processes which incorporate whole-of-life
  operating and service delivery costs based on
  comprehensive planning and upfront identification
  of the full scope of the project.

44
Key Recommendations

• Project Delivery
• Agreements with Service Providers
• Public sector entities establish service level
  agreements and contracts with service providers
  which address the objectives and scope of the
  project, performance expectations and reporting
  requirements.

45
Key Recommendations

• Project Reviews
• Project Reviews
• Public sector entities conduct post project
  evaluations and building performance reviews as a
  standard process for capital works projects.

46
Key Recommendations

• Project Governance
• Risk Management
• Public sector entities ensure that all risks in
  relation to their overall program and individual
  projects are identified in the early stage of
  planning, together with strategies for the
  ongoing management of these risks.

47
Key Recommendations

• Quality Assurance
• Public sector entities conduct periodic
  independent quality assurance reviews over
  capital works activities to ensure that agreed
  processes are being followed and that delivery
  and construction meets the specified
  requirements.

48
Key Recommendations

• Records Management
• Public sector entities ensure that key project
  documentation is centrally managed and securely
  maintained in order to provide a clear management
  trail of decisions made.

49
Queensland Public Accounts Committee

• The role of the Public Accounts Committee is to
  assess the integrity, economy, efficiency and
  effectiveness of Government financial
  management. It achieves this by
• examining Government financial documents and
  considering the reports of the Auditor-General.

50
PAC Hearings
51
PAC Areas of Interest

• Issues raised in the Audit-Generals Report that
  were considered critical?
• Reasons for the successes and failures of the
  Capital Works Systems?
• Correlation between the size and complexity of
  the departments capital works program and the
  audit findings?
• The overall impact on the departments that had no
  documented program or project objectives for
  their capital works program?

52
PAC Areas of Interest

• The impact (positive or negative) of departments
  relationship with respect to capital works with
  the Department of Public Works?
• The reason behind cost overruns? Were these
  related to the accuracy of the costings
  themselves or rather the scope of the project,
  and that the cost overruns were because project
  costings did not cover everything?

53
PAC Areas of Interest

• The timelines for addressing findings and
  recommendations within AG Report.
• Reason for noncompliance with government policy
  (Capital Works Management Framework).
• What scope is there for increased reporting
  external to identify capital works cost
  overruns?

54
PAC Report Areas of Focus

• Sound strategic planning.
• Project evaluations.
• Compliance with Government Policy.
• Appropriate project planning and scoping to
  minimise cost overruns.
• Closer inter-agency relationships.

55
Conclusion

• The recent rapid growth of the Queensland economy
  has created significant demand upon the States
  infrastructure and QAO considers that the
  recommendations contained within this report will
  be of increasing significance in the current
  climate. This is particularly the case with the
  planning and service delivery recommendations as
  well as compliance with the Capital Works
  Management Framework.
• This growth has lead to the development of new
  infrastructure strategies, frameworks, policies
  and guidelines creates compliance obligations for
  public sector agencies and QAO will be examining
  some of these aspects as part of our annual
  financial and compliance audits.

56
Whole-of-Government Risk Assessment and
Management Review

• Sandra Heidrich
• Acting Director of Audit
• Queensland Audit Office

57
Overview

• Background
• Progress update
• Surveys
• Audit visits
• Current focus
• Audit report
• Better Practice Guidelines
• References

58
Background

• Request from Department of the Premier and
  Cabinet
• Evolution of Risk Management

59
Evolution of Risk Management
Strategy
Enterprise Risk Management
Management

• Focus Integrated Enterprise-wide Risk
• Scope Align strategy, processes, people,
  technology and knowledge on an enterprise-wide
  basis

Business Risk Management
Operations

• Focus Business Risk
• Scope Business managers accountable
  (risk-by-risk)

Risk Management
Financial

• Focus Financial and hazard risks and internal
  controls
• Scope Treasury, insurance and operations involved

Source Arthur Andersen
60
Progress update

• Surveys
• Completed by all budget sector agencies
• Reviewed by QAO financial and compliance audit
• Follow up queries on specific issues
• Data aggregated

61
Progress update

• Audit visits
• Purpose of the visits
• 9 agencies visited
• Preliminary comments
• Various levels of maturity
• Overall systems to manage risk at operational and
  project level seem adequate

62
Current focus

• Audit report
• Overview of current risk management practices in
  the budget sector
• Focus on three broad areas
• WoG Risk Management Framework
• Corporate Governance and Culture
• Risk Management Systems and Processes
• Outcome Better Practice Guidelines

63
Better practice guidelines

• Focused on risk management at strategic level
• High level principles
• Case studies

64
References

• AS/NZS 43602004
• Enterprise Risk Management Integrated
  Framework, COSO (2004)
• Risk Improving governments capability to handle
  risk and uncertainty, Strategy Unit UK (2002)
• Management of Risk - Principles and Concepts,
  Treasury UK (2004)
• Treasury Board of Canada Secretariat
• www.tbs.gc.ca/pubs_pol/dcgpubs/RiskManagement/sigl
  ist_e.asp
• Enterprise-wide risk management Better Practice
  Guide for the public sector, CPA Australia (2002)
• Principles of Good Corporate Governance and Best
  Practice Recommendations, ASX (2003)

65
Looking Ahead

• Terry Campbell
• Assistant Auditor-General
• Queensland Audit Office

66
Overview

• Audits underway
• Extension to PMSA mandate
• Follow up audits
• Planned audits

67
Audits underway

• Risk Management Review
• To be completed June/July
• Output Performance Reporting
• Report to be tabled 7 August 2007

68
Extended PMSA Mandate

• Extension of mandate over performance management
  systems audits came into effect on 1 January
  2007.
• Approved through amendments to the Financial
  Administration and Audit Act 1977.
• Now able to look at whether information that
  agencies are reporting is part of an appropriate
  and relevant suite of measures that establishes a
  clear link between what is being measured and
  what the agency and government wants to achieve
  for the Queensland community.

69
PMSA Role

• To verify to Parliament and the community that
  the performance information publicly reported is
  credible and meaningful.
• To identify and show where these key performance
  management risks lie across the public sector and
  report these findings in a balanced and
  responsible manner to Parliament.

70
Defining the terms

• Consider that key accountability documents such
  as the MPS and the Annual report are only
  effective if they report against performance
  measures that
• Report the right things (relevance and
  appropriateness)
• Report fairly (are accurate and reliable)
• Fairly represent (are meaningful to any group or
  individual who can affect, or is affected by, the
  achievement of any agencys purpose), and
• Are audited and evaluated to provide assurance on
  relevance, appropriateness and accuracy.

71
Follow up audits

• Important element of accountability process.
• Report extent to which agencies have implemented
  recommendations of previous audit reports.
• Keep Parliament informed of progressive
  improvements and current challenges in areas that
  have previously been subject to scrutiny.

72
Current follow up audits

• Report Nos 3 and 5 for 2005 on Output Performance
  Reporting.
• Report No. 2 for 2005 on Queensland Disaster
  Management System.

73
Planned audits

• Scoping underway in relation to
• Workforce skills
• Water infrastructure planning.

74
Questions
75
Thank you for attending

• We welcome your feedback on this session. Please
  complete the yellow evaluation form and leave at
  the exits.