Water Implementation Projects In Rural North India: Evaluating implementation

1
Water Implementation Projects In Rural North
India Evaluating implementation on the ground

• Katharine Owens, University of Hartford,
• Politics and Government
• Marcia Hughes, University of Hartford,
• Center for Social Research

2
Overview of Presentation

• Project efforts to date
• Evaluation, implementation, and collaboration
• Assumptions of this evaluation work and inherent
  challenges of project efforts
• Evaluation model and methodology Contextual
  Interaction Theory
• Analyses, findings, and uses of the model
• Recommendations and conclusion

3
Projects in Abheypur, India

• University of Hartford student-faculty
  service-learning teams
• Professional and student chapters of Engineers
  without Borders
• Interdisciplinary projects engineering, social
  science, and education/outreach projects focusing
  on water.

4
On the ground in Rural India
Abheypur
Delhi
Gurgaon
Abheypur
Pathways
5
Abheypur, India Project Team
Abheypur, India
6
The Projects

• January 2008 Installation of a solar powered
  well pump and water storage system
• August 2008 Extension of the water supply to a
  poor section of the village
• January 2009 Implementation of a
  rainwater-harvesting system
• October 2010 Installation of a potable water
  supply well and water storage tank for poorest
  community January 2010 Installation of a grey
  water drainage improvement project for the
  Potters Village
• January 2011 Installation of a grey water
  drainage improvement project on main road of the
  village

7
Abheypur, Haryana
October 2009 tank location

• Well
• Tanks

• 2 tanks
• Piped to girls school

2010 Soak Pit
2011 Soak Pit

• Solar panels
• 2 wells
• 4 tanks
• Rainwater harvesting

8
(No Transcript)
9
Main Road Drainage Problem
10
(No Transcript)
11
Design for Soak Pit Implemented in 2011
12
Potters Village Drainage Problem
13
BEFORE
14
Potters Village - Soak Pit Design Jan 2010
Two layers (6) of bricks are used for soil
structural support to keep soak pit open. Thus,
excavated diameter is 1 wider and longer than
internal dimensions listed.
Forebay Soak Pit
4
4
Overflow to swale during for rain events
Greywater Inflow
4
Stones and rocks to reduce water velocity
Brick Weir (Width 3)
Forebay Depth 1.5 Soak Pit Depth 4 Forebay
and Soak Pit covered with slate
15
AFTER
16
Implementation

• When political actors desire change in society,
  they develop and implement programs to address a
  perceived problem.
• Researchers focus on the implementation stage of
  the policy process as a point where intentions
  and outcomes diverge
• Two different processes Developing a
  program/policy versus implementing a
  project/program/policy

17
Merging evaluation and implementation

• Program evaluators seek to quantify and catalog
  the successes and failures of programs and
  policies.
• Implementation policy analysts evaluates this
  stage in the process to better understand
  participants roles and any impediments to
  change.
• We seek to infuse evaluation with implementation
  analysis methodology.

18
Merging the two

• Evaluation researchers find its not enough to
  know whether outcomes are achieved, its
  important to understand how.
• Using an implementation analysis tool in
  evaluation can answer questions about how people
  come together and work towards achieving a
  project (or fail to).
• Scholars call for responsive, contextual,
  flexible, adaptive, multidisciplinary, and
  mixed-methods approaches, particularly when
  considering environmental programs and policies
  (Preskill, 2009, p. 99).

19
Collaboration is an imperative (Gadja, 2004)

• Being solo does not work for most of the problems
  that need addressing
• Need shared efforts to achieve goals that would
  not otherwise be attainable working
  independently.
• More and more, collaboration is becoming the
  method for addressing complex social problems

20
Understanding collaboration

• Wostl et al. (2007)
• In managing water resources in particular, a
  paradigm shift is needed, one that recognizes the
  importance of stakeholder involvement and
  collaboration.
• Water communal and imperative

21
Understanding collaboration

• One goal To develop communities of practice.
• Would we recognize if we were not working
  collaboratively with our Indian partners?
• Should we trust our own conceptions, or evaluate
  the process to confirm our goals of
  collaboration?

22
What does collaboration look like?The Practice
of Collaboration

• Gajda and Koliba (2007) describe key
  characteristics of interpersonal collaboration to
  be
• a shared purpose
• dialogue
• decision-making
• action, and
• evaluation (p. 29).

23
Water Implementation Projects in India Inherent
Challenges

• Working across cultures
• Across languages
• Across disciplines (engineers and social
  scientists)
• Across class (American middle-class and Indian
  impoverished villagers)
• Caste distinctions remain present in the village
• Gender distinctions (American women in positions
  of leadership, which may be atypical)
• There is short period to build trust among
  partners, implement new projects, and manage and
  evaluate previous projects.

24
This evaluation work

• We work to accomplish project implementation as
  well as outcome evaluation in India.
• We find it important to engender true
  collaboration with India partners.
• We use an implementation analysis theory to
  evaluate these water projects.

25
Contextual Interaction Theory

• finds project participants and other stakeholders
  can influence implementation in critical ways.
• analyzes collaboration between participants based
  on motivation, knowledge, and power, to shed
  light on how projects are implemented.
• produces a prediction about interaction based on
  these core participant characteristics

26
Contextual interaction Theory

• allows the emphasis on implementation (the how)
  as called for by evaluation researchers.
• can provide evidence to those on the ground
  about their effectiveness in implementation.
• allows large-n studies that enable researchers to
  find patterns in large data sets and enables
  multi-year studies.

27
Contextual Interaction Theory

• The motivation, knowledge, and power of the
  implementer and target are the focus of the
  assessment.
• Implementer the student-faculty team, and a
  loose coalition including Navjoti, and the
  professional Engineers without Borders chapter
• Target local villagers, including those residing
  near the project and those taking an interest in
  the implementation of the project.

28
Research Questions

• How does motivation, knowledge, and power among
  the different players affect implementation of
  the project?
• What is the relationship between these
  characteristics among the people on the ground
  and the outcomes of the implementation process?
• How is the project plan or design implemented
  through collaboration processes?

29
Methods

• Faculty-student trip to Abheypur, India in
  January 2010.
• 14 interviews using a semi-structured interview
  instrument.
• Questions investigate the core elements of the
  theory (motivation, knowledge, and power)
• We asked slightly different questions of the
  American and Indian counterparts as reflected
  their roles in the process.

30
Independent variables

• Motivation
• Knowledge
• Power

31
Dependent variables

• Active cooperation
• Passive cooperation
• Forced cooperation
• Opposition
• Obstruction
• Joint learning
• No interaction
• Policy learning

32
Analysis
1 M scale((proportion)-.50) 2, scale of -1.0
to 1.0 2 I and P scales (proportion), scale
of 0.00 to 1.00 3 n/a participant did not
answer questions on this concept.
Role Motivation M scale1  Category Knowledge K scale2 Power  P scale
Implementer Faculty-student team members () 5 of 5 1.00 positive () 4 of 4 1.00 () 2 of 4 0.50
Implementer Faculty-student team members () 2 of 4 0.00 neutral () 3 of 6 0.50 () 0 of 2 0.00
Implementer Faculty-student team members () 2 of 2 1.00 positive () 4 of 6 0.67 () 0 of 2 0.00
Implementer Faculty-student team members () 1 of 2 0.00 neutral () 4 of 5 0.80 () 1 of 2 0.50
Implementer aggregate () 10 of 13 0.538462 positive () 15 of 21 0.7142857 ()3 of 10 0.30
Target Villagers () 3 of 3 1.00 positive () 2 of 2 1.00 () 0 of 3 0.00
Target Villagers () 2 of 3 0.33 positive () 1 of 1 1.00 () 0 of 2 0.00
Target Villagers () 2 of 2 1.00 positive () 2 of 2 1.00 () 0 of 2 0.00
Target Villagers () 3 of 3 1.00 positive () 2 of 2 1.00 () 0 of 1 0.00
Target Villagers () 1 of 2 0.00 neutral n/a3 n/a () 0 of 1 0.00
Target Villagers () 1 of 2 0.00 neutral () 1 of 1 1.00 () 0 of 2 0.00
Target Villagers () 2 of 2 1.00 positive n/a n/a n/a n/a
 Target aggregate () 14 of 17 0.647059 positive () 8 of 8 1.00 () 0 of 11 0.00


33
Analysis
34
Qualitative assessment

• Implementers have some doubts about the work,
  their role, and the role of the target
  participants in the process.
• "but you know sometimes I wonder maybe there
  might be actually better things we could be doing
  with our time."
• "I think with this project we weren't
  communicating with them before we got there, they
  didn't know our plan.
• New understandings of caste subgroups and inter
  and intra group dynamics

35
Analysis

• What is the potential for other scenarios to
  emerge?
• The strong imbalance of power does not come into
  consideration when both actor groups are in favor
  of a project.
• But how might this impact implementation if one
  actor was no longer in favor of implementation?

36
If the target lacked motivation
37
If the implementer lacked motivation
38
Evaluating our collaboration
39
Recommendations (February 2010)

• Community-driven projects based on a formalized
  decision-making process.
• establish a shared purpose/role among EWB
  interdisciplinary team
• Always consider the caste/subcaste/gender/poverty
  issues and related power/resources, knowledge,
  and motivation among different groups

40
Recommendations (February 2010)

• Make collaboration a goal
• Are all stakeholders involved?
• Is there a shared purpose and understanding of
  the project?
• Is there equal decision-making? Has the community
  been involved and/or surveyed for their feedback?
  
• Is there local leadership for the project (i.e.,
  in the village)?
• Expect and plan for disagreement and conflict.

41
Recommendations (February 2010)

• Streamline the collaborative process
• Develop project ideas with community leaders and
  Navjyoti.
• Navjyoti can survey/scan community for
  input/feedback on project ideas.
• Navjyoti can serve as link between EWB and
  village and district leadership (i.e., for
  communicating project design development, and
  planning).
• Fine tune plan and develop budget-tasks and
  responsibilities (prior to starting project).

42
Recommendations (February 2010)

• To make previous projects sustainable
• Monitoring
• Planning
• modifications
• Monitor systems to ensure proper use and
  satisfaction with
• system(s),
• community leadership, and
• overall quality of water

43
Conclusions

• It is both a goal and an obligation to work in a
  collaborative way with local groups in India.
• Using the theory we can gain insight into not
  only the outcome of implementation (cooperation),
  but also provide context for how and why
  participants are cooperating.

44
Conclusions and Next Steps

• Our analysis also sheds light on how the people
  involved in this project influence it.
• It allowed us to set meaningful goals for
  collaboration and partnership
• Next Steps Analysis of 2011 data