Quality Audit Tool for Managing Social Performance

1
Quality Audit Toolfor Managing Social
Performance
2
Acronyms Used

• QAT Quality Audit Tool
• ECA region of Eastern Europe and Central Asia
• MFI microfinance institution (MF Banks,
  non-banking financial institutions,
  non-governmental organizations)
• SP Social Performance
• SPM Social Performance Management
• SR social responsibility

3
Agenda

• QAT Overview
• QAT Process
• QAT Attributes
• QAT vs. Other Tools
• QAT Implementation

4
Agenda

• QAT Overview
• QAT Process
• QAT Attributes
• QAT vs. Other Tools
• QAT Implementation

5
Rationale ECA Region Context
Market trends
Social Performance reality
Context

• Since beginning in 1997 focus and pressure on
  MFIs self-sustainability
• Withdrawal of donors to less developed countries
• New comers commercial investors (i.e. Deutche
  Bank, Raiffaisen Bank)
• Post soviet negative connotation of social
  for free

• Commercialization of MFIs
• Mission drift observed
• Still huge interest in impact, but no resources
  to measure

• Difficulties in promoting Social Performance
• No commitment to social goals at the level of
  operations
• Interest in easy and effective tool bringing
  quick measurable results

6
QAT Purpose
in order to
QAT aims at

• Assessing status and effectiveness of systems in
  managing SP
• Helping MFI realize its strengths and weaknesses
  in SPM
• Helping MFI prioritize actions for improvement of
  their SPM
• Ensuring institution-wide understanding and buy-in

• To ensure MFI is on track how effective are
  your systems in aiming at achieving your
  mission?
• To optimize resource investment build on what
  you have and invest in what you really need
• To prevent MFI from doing everything at once and
  burn out shortly lets improve step by step
  and look for quick wins
• To create enabling environment for change to
  prevent SPM be one person project

7
QAT Target Markets

• MFIs concerned about their Social Performance and
  quality of their management systems for SP
• MFIs planning social rating exercise and willing
  to prepare themselves
• Donors / Investors / Partners willing to support
  their partnering institutions in improving their
  SPM

8
SP Dimensions Assessed by QAT
Intent and design
SP information system
Management systems
9
SPTF Dimensions in QAT
10
QAT Uses for MFI Manager

• To check if an MFI is at right track towards
  their mission achievement
• To identify strengths supporting their SPM
• To identify weakness gaps and areas for
  improving their SPM
• To use prioritized initiatives for developing
  action plan for improvement, allowing step by
  step SPM institutionalization
• To buy in staff at all levels to build
  understanding SPM is overarching theme rather
  than project or tool
• To monitor progress in SPM institutionalization
  (in case of repeated audit)
• To prepare for social rating

11
QAT Uses for MFI Manager - Examples

• Horizonti QAT lessons learnt
• This was very important for we did the audit
  before implementing anything conducting the
  audit we knew the gaps where we need to pay
  more attention.
• One of the outcomes we do know what we are
  doing, we are targeting the right people and
  doing a good job, but we need to work on systems
  to support this.
• We saw that the MIS cannot support performance,
  so we will have a new one improved.
• We collect a lot of information which can be used
  for SP indicators.
• ED, Horizonti, Macedonia

• Zene za Zene QAT next steps
• Define more specific objectives for better
  understanding what it means for the terms of SP?
• To experiment with MIS reports content
• To improve internal communication
• To measure drop-outs
• To use client satisfaction form across the
  institution
• SPM Champion, Zene za Zene, BH

12
QAT Uses for Donor / Investor / Partner

• To help MFI improve in SPM through diagnosing
  strengths and weaknesses and developing action
  plan
• To optimize its support through financing or
  helping implement the initiatives from action
  plan
• To better understand the institution, its mission
  orientation and rational behind its translation
  to action
• To monitor progress in SPM institutionalization
  (in case of repeated audit)

13
QAT Desired Impact MFI Level

• Outcome - setting the stage for change process
• Clearer operational understanding of SPM
• Clear understanding by staff of what needs to be
  done to improve SPM
• Internal champion and sponsor identified
  infrastructure for change established
• Better communication of social orientation and
  required support to stakeholders
• Quick wins implemented
• Short term impact
• Process of SPM improvement kick-off
• Consistent effort of MFI to improve SPM in a
  manageable steps
• There is on-going buy-in of staff, leadership of
  champion and support of senior management and
  board for implemented changes
• Medium term impact
• Internal systems are improved to be better
  aligned with double bottom line goals
• Long term impact expected
• Institution gets all aligned to double bottom
  line and results are improved

14
QAT Impact Examples From MFIs
CARD, the Philippines
Gap description of the target group was vague
and mixed with eligibility criteria for the loan
Quick win through a series of half day workshops
defines what is meant by saying we want to reach
economically and socially challenged families
and comes up with the indicators based on housing
materials, asset base, etc.
Moznosti, Macedonia
Gap At fast growth, new staff dont understand
and commit to mission
Quick win re-design of training program for
newly hired staff to ensure mission perspective
incorporation
Micro Development Fund, Serbia
Gap collects lot of information, which is not
used later on
Quick win review the new set of MIX SP
indicators to see which information they can
start using immediately
More examples of MFIs quick wins in MFC
Newsletter Issue 1/2008
15
QAT Desired Impact

• Investor / Donor / Supporter level
• Increased number of partners refocus from
  assessing SP to improving SPM ? more supportive
  and longer term cooperation on SPM improvement
• More informed partners make better decision in
  selecting more likeminded partners
• More efficient allocation of resources to
  likeminded MFIs and activities that can bring
  greatest value in improving SPM at MFI
• Industry level
• More MFIs aware of their SPM status and
  effectiveness
• More MFIs improve in their SPM
• Less mission drift
• Better public image of Microfinance

16
QAT Benefits for Various Stakeholders

• MFI Managers
• Clear current picture of SPM systems quality
• Good basis for effective implementation plan
• Efforts oriented on mission acknowledged
• Staff understands better and is bought-in through
  discussion about SPM
• Allows improving before future social rating
• MFI staff
• Better understanding of how their tasks fit into
  social goals and strategy
• Donors / investors / supporters
• Better understanding of MFIs commitment to
  mission and its translation into practice
• Assessment of MFIs SPM quality
• Effective identification of MFIs needs in terms
  of supporting their SPM institutionalization

17
QAT Uses Extracts from 5th SPM Working Group
Meeting Minutes

• In March 2008, the representatives of 10
  organizations members of MFC facilitated SPM
  Working Group, met in Warsaw to discuss lessons
  learnt in SPM institutionalization. Among other
  subjects they discussed QAT uses
• It can be used before the process of SPM
  implementation it is nice to have gaps
• Could be used for process evaluation a good
  diagnostic tool
• Could be useful for taking on board those who
  were not directly involved e.g. IT people
• Gives an understanding where the gaps are and
  what actions need to be taken
• The format is very useful the SPM team might be
  biased but through interviews one gets more
  objective information
• Could be used multiple times especially before
  the strategic planning
• Provides awareness of different opinions and
  perceptions
• Stimulates additional discussions at the
  institutions on SPM
• Gives better understanding of HR needs where
  there are gaps in assumed and actual way the
  processes work
• Remind and refresh the mission and social
  objectives
• Reveals information flow bottlenecks

18
Agenda

• QAT Overview
• QAT Process
• QAT Attributes
• QAT vs. Other Tools
• QAT Implementation

19
Agenda

• QAT Overview
• QAT Process
• QAT Attributes
• QAT vs. Other Tools
• QAT Implementation

20
4 Steps in QAT
0,5 day
2 - 3 days
1-2 days
0,5 day
1 day
1 day
Preparation
Gap analysis
Follow up interviews
Audit panel
Final report
Analysis draft report

• ED
• Senior management (Finance, Operations, HR,
  Marketing, )

• Board
• Senior management
• Internal auditor
• MIS staff
• Research staff
• Regional level
• Branch level
• LO
• Clients

• All

Staff participating
Total time from 6 up to 8 days
21
External Internal Assessor
QAT is implemented by the team of two people
external and internal assessor
22
QAT Implementation Steps (1)

• Gap analysis presents detailed questions which
  are used to assess SPM sets up a base for more
  in-depth follow-up
• Source ED and senior management
• In-depth follow-up gathers more detailed
  information to verify and better understand gap
  analysis checks status and effectiveness of
  systems
• Source Interviews with senior management, staff
  clients review of documents and internal
  reports and information flows.

23
QAT Implementation Steps (2)

• Analysis and draft report combine information
  and draft report highlighting strengths and
  weaknesses of issues in gap analysis it captures
  perspectives of different stakeholders to be
  discussed during audit panel.
• Analysis include
• Qualitative data analysis using triangulation of
  different information sources (interviews with
  different staff, focus groups with clients,
  review of written documents)
• Tally sheet is used to identify common themes and
  divergences around strengths and weaknesses of
  MFI systems, notes from the interviews are
  reviewed for evidence and rationale behind
  emerging themes
• Draft report is written addressing strengths,
  weaknesses, evidence and issues to further
  discuss/explore that serve as basis for
  discussion with representatives of different
  stakeholders during an audit panel.

24
QAT Implementation Steps (3)

• Audit panel Presentation of findings to key
  organizational stakeholders, including staff from
  all levels of organization and clients, where
  appropriate.
• Audit panel allows for further verification of
  the audit results and finalize the report
• The initiatives for addressing gaps are
  identified and prioritized in order to develop
  action plan for SPM improvement
• Final report finalized based on audit panel
  discussion
• Includes prioritized activities to be undertaken
  by MFI in order to improve SPM by addressing the
  identified gaps and by building on identified
  strengths

25
Source of Information and its Validation
26
Data Collection Methodology

• Gap analysis questions basic set of questions
  defining the framework of SPM assessment
• i.e. do you seek to be socially responsible
  organization in relation to your target clients?
• Status questions questions validating the status
  of SPM components their presence in the
  institution
• Asked to Loan Officer How do you communicate
  interest rate to clients? How do you ensure they
  understand?
• Effectiveness questions
• Asked to clients How do you learn about interest
  rate? How do you know what is it?

27
QAT questions examples
Source QAT Handbook
28
QAT Report Structure (1)

• Report is divided according to dimensions and/or
  sub-dimensions
• Each dimension / sub-dimension includes
• to be discussed during audit panel
• List of institutional strengths supported by
  evidence
• List of institutional weaknesses supported by
  evidence
• Points for discussion when there are different
  and not coherent opinions and evidence, they can
  be clarified and agreed during audit panel
• after audit panel
• key elements of the action plan

29
QAT Report Structure (2)
30
Agenda

• QAT Overview
• QAT Process
• QAT Attributes
• QAT vs. Other Tools
• QAT Implementation

31
Agenda

• QAT Overview
• QAT Process
• QAT Attributes
• QAT vs. Other Tools
• QAT Implementation

32
QAT Attributes - Time

• Depends on skills and experience of assessor
• With external facilitator the process is more
  intensive and shorter
• Internal Assessor can adjust duration to the
  internal dynamics

Duration
Process steps

• Preparation (documents review)
• Gap analysis
• Individual interviews
• Focus Group Discussions with Loan Officers and
  Clients
• Analysis
• Audit panel
• Final report

• 1 2 days
• 0,5 day
• 1,5 2 days
• 1 day
• 0,5 -1 day
• 0,5 days
• 1 day

TOTAL 6 8 days
33
QAT Attributes Cost

• Staff time
• Internal Assessor
• Staff in interviews
• Staff in audit panel
• Travel to branches for FGDs
• Snacks for FGDs

• Includes
• 6 8 days
• 1,5 h per person
• 3 h per person
• Travel costs
• Costs of drinks and snacks

MFI costs

• External facilitator time
• Lodging
• Travel

• 6 days daily rate
• 6 nights daily allowance
• Cost of plane ticket

External facilitator
MFC estimated cost for external assessment 6000
USD staff time cost
34
QAT Attributes Breadth of Functionality
35
QAT Attributes Infrastructure and Resource
Requirement

• Resources required
• Staff time
• Funding for external facilitator budget
• Infrastructure required
• Computer for assessors, recorder
• Separate room for conducting interviews
  (preventing noise, securing proper concentration)
• Convenient room for 2 FGDs (with Clients and Loan
  Officers)
• Transportation to branch (if needed)
• Conference room for holding audit panel
  (including LCD projector, flipcharts, markers)

36
QAT Attributes Ease of Use

• In case of external assessment
• MFI needs only to set up logistics and select
  internal assessor
• Additional benefit MFI is trained on how to
  conduct the assessment to do it independently
  next time
• Challenge securing the funding
• In case of self-assessment
• Pre-assessment training is required
• Timing adjusted to the internal dynamics of MFI
• Challenge the selection of internal assessor
  must be more careful

37
QAT Attributes Customer Satisfaction

• Assessing both strengths and weaknesses makes the
  audience appreciate the objective assessment
  they are motivated to improve as their up to date
  efforts to achieve the mission are acknowledged
• Involvement of many stakeholders into process and
  allowing them commenting during audit panel make
  them owe the findings in opposite to external
  findings (external assessor becomes perceived
  rather as facilitator)
• Developed action plan reflects the real needs and
  does not irrationally overburden staff
• Clear and easy to follow structure of the report
  makes it a handy document for further work

38
QAT Attributes Customer Satisfaction in Citations

• I am pleasantly surprised how everything is now
  clear the role and the goals of SPM
• External assistance in the audit is very helpful.
  I wonder whether to us people would say what the
  external auditor found out. I have a feeling they
  tell us what they think we want to hear.
• It was good that loan officers and branch staff
  were included in this audit. They gave a needed
  perspective for the understanding of the issues
  they face in their work. The fact that all staff
  was included means we will have full support for
  the changes
• I am the SPM team member. During this week it
  became clear that we did a lot of stuff,
  discussed and decided on many points. I now see
  what will be the result of the entire SPM effort.
  
• Social audit made me realize that we are already
  working on many aspects of SP in our
  organization, but have not given it the right
  emphasis
• I am now better aware of the importance of the
  right loan for the client. I will pay more
  attention to how I do my work, how the loan is
  used and whether the client is advancing. This is
  giving me an encouragement for my future work
• Partner, Bosnia and Herzegovina

• In our case, this tool revealed many weak points
  in the systems and management processes and
  resulted in creating an institutional strategy
  with a holistic approach
• AgroInvest, Montenegro, Serbia

39
Agenda

• QAT Overview
• QAT Process
• QAT Attributes
• QAT vs. Other Tools
• QAT Implementation

40
Agenda

• QAT Overview
• QAT Process
• QAT Attributes
• QAT vs. Other Tools
• QAT Implementation

41
QAT vs. Other Tools (1)
Compliance audits social accounting by Social
Audit Network (UK)
Social Performance Audit Tool (SPA) by USAID
42
QAT vs. Other Tools (2)
SOCIAL Tool by ACCION
Social Performance Indicators by CERISE
43
MFIs Implementing QAT Worldwide

• ECA region
• AgroInvest and MDF, Serbia
• Aregak, Armenia
• FinDev, AzerCredit and Viator, Azerbaijan
• Horizonti and Moznosti, Macedonia
• Partner and Zene za Zene, Bosnia and Herzegovina
• The Philippines
• ASHI, CARD
• MENA Region
• Tamweelcom and DEF, Jordan
• Till end of 2008 ARDI and AMOS, Morocco

44
Contact Details

• For more information on QAT, please contact
• Ewa Bankowska, Project Coordinator
• E-mail Ewa_at_mfc.org.pl
• Phone [phone]
• Skype bankowska