Cultural Competency and Evaluation: An Overview

1
Cultural Competency and Evaluation An Overview

• Ranjana Damle
• Albuquerque Public Schools
• New Mexico Evaluators
• 2009 Summer Conference

2
Outline of the Presentation

• Purposes of evaluation
• Brief discussion of evaluation theory
• Types of evaluation new discourse, trends
  efforts to live up to evaluations promise
• Context of evaluation
• Social construction of reality common
  assumptions and blind spots
• Issues of power and social justice
• Culture, cultural competency, culturally
  competent evaluation
• A case study that ties in many of these points

3
Evaluation Purposes

• Accountability and program improvement
• Policy development and informed decisions
• Systematic inquiry to generate knowledge

4
Three Main Streams in Evaluation Theory

• Methods creating knowledge using rigorous
  research techniques
• Valuing evaluators discern and place a value on
  their findings favors decision makers
• Use evaluations intended to inform decision
  making various stakeholders perspective
  incorporated to increase use

Alkin Christie 2004
5
Evolution of Evaluation

• Participatory a continuum minimal to full
  participation of the stakeholders a buy-in for
  the evaluation in the organization Bradley
  Cousins
• Collaborative to increase the chance of
  evaluation use
• Utilization-Focused Michael Patton
• Responsive Robert Stake Stakeholder
  perspective needs to be represented In the
  evaluation-extends from decision-makers to a
  variety of stakeholders, however, evaluator is
  the final authority

6
Use for the Intended Users

• Empowerment David Fetterman - stakeholder
  controlled self-determination as a goal of the
  approach
• Advocacy democratic Jennifer Greene
  (Evaluators inevitably take sides.)
• Social Justice Ernie House Evaluation is
  never value neutral and it should tilt in the
  direction of the poor and the powerless.
• Cultural Competence in Evaluation Karen
  Kirkhart, also Rodney Hopson, Saumitra SenGupta,
  Melva Thompson-Robinson, many others

7
Context of Evaluation

• Political, economic, social, cultural
• Diversity - age, sexual orientation, differential
  abilities, race and ethnicity, gender, immigrant
  status, etc.
• Less discussed differences - access to power,
  education socioeconomic status, class living
  situation

8
Social Theory

• Social construction of reality
• There is a social dimension to human behavior
• We create, attach meaning to symbols
• Reality is not absolute but as individuals or
  groups define it
• Cultural relativity examples, adult son living
  at home, parental participation in schools

9
Some Common Observations, Experiences across
Societies

• We relate to people that share our experiences
• In-groups and out-groups embracing in-group
  norms and beliefs, critical of out-groups
• Middle class bias in education and other social
  institutions

10
Invisible, Hidden, Impactful

• Privilege taken for granted by those who have
  it
• Blindness towards the underprivileged e.g.
  caste societies
• Dominant norms - treatment of else as something
  deficit

11
Evaluation and Issues of Power, Equity, Social
Justice

• Unequal power relations between different
  stakeholders
• Whose interests are represented in the
  evaluation?
• Objectivity and neutrality versus inherent
  biases, e.g. favoring the management

12
Defining Culture

• Food, music, clothes
• Customs, norms, beliefs, myths, social
  institutions, language, other symbols, traits
• Values - While there are some universal values,
  there is variation across cultural groups.

13
Cultural Competency

• Being effective in cross-cultural situations and
  able to relate to many cultures
• As an evaluator, being able to recognize the
  culture-specific strengths and weaknesses
• A distinction to be noted
• Evaluating cultural competence
• Cultural competence in evaluation

14
Cultural Competency in Programs Evaluation

• Cultural competency in program development
• Respect, appreciation, flexibility, recognition
  of values and needs although different from ours
  programs likely to be more effective if
  culturally relevant, e.g. population control
  programs
• Cultural competency in evaluation
• Culturally responsive evaluations
• Ethical issues, equity, awareness of the power
  imbalance, identifying program elements that seem
  to make programs work for non-dominant cultures

15
Towards Successful Evaluations

• Cultural competency
• Is essential in study design, validity of
  measurements, and data collection
• Leads to pertinent questions what strategies in
  a successful program served the needs of the
  (minority) clients and why so
• Leads to a greater buy-in, evaluation findings
  are applied towards improving programs

16
A Case of Lofty Goals, Unilateral Program
Design, Wasted Effort

• The Myth of Population Control by Mahmood Mamdani
  (1973)
• Program was designed to educate the villagers
  about family planning but without taking in to
  account their experience, perceptions, values,
  and realities of their lifes challenges
• Cultural competency issue of rationality
• Validity of measurement closely tied to cultural
  competency what was measured and how
• Outcome and lessons

17

• Contact Information
• Ranjana Damle
• Research and Evaluation
• 872-6801
• damle_at_aps.edu