About this presentation For the Reviewer

1
About this presentation For the Reviewer

• I am a member NAE4-HA Research Committee
• In 2006 we were invited to submit a proposal for
  two evaluation workshops (basic/intermediate
  advanced intermediate) for the 2007 conference.
• I was the leader of the team that planned and
  presented a proposal for the basic/intermediate
  workshop
• It was accepted and a team of five
  individualswith me as the facilitator and a
  co-presenterpresented a workshop at the 2007
  NAE4-HA Conference in Atlanta.
• What follows is my section of the workshop on
  qualitative and questioning techniques for
  evaluation

2
Questions Can Be the Answer!

• Furthering understanding and abilities in the
  magical land of qualitative evaluation
• 2007 NAE4-HA Conference Atlanta, Georgia
• G. F. Varrella, Ph.D.,
• WSU Extension-Spokane County

3
Our Discussion

• The value and place of qualitative work
• Two methods
• Basic considerations for generating questions
• A bit of background on the art of asking
  questionsand still be tuned into your goals,
  content, and context!

4
Some days it may feel like all we are every asked
is
Higher
5

• ( and here comes the sales pitch)

• Qualitative evaluation research is flexible
• You can free yourself from recipeswhich is
  especially appealing if you love the topicdraw
  on your expertise and enthusiasm!
• With a little freedom comes the option to start
  at a level you that you choosethat fits your
  audience (making you more nimble)

6

• As the process of inquiry continues, you have the
  right to revise and rewrite your rules and
  reexamine your suppositions
• As the inquiry continues, you will discover which
  methodologies work best (i.e., qual., quant.,
  mixed)
• (Freeman, et al., 2007)

7
Higher!

• You are asking open-ended questions
• Seeking the facts from traditional and
  emotional/personal perspectives
• and you are more able to align your evaluative
  inquiries and questions to your stated outcomes

8
Interviews

• At the most basic level, are conversations.
• That,
• attempt to understand points of view,
• unfold the meaning of experiences,
• uncover changes in individuals, groups and
  communities.
• From Hrabik, Leahy, Minter, Westover

9
Guides for Interviews

• Be personal and unobtrusive
• Obtain direct feedback
• Seek understanding
• dig deeper
• Observe behaviors and reactions
• Obtain rich, detailed data
• To be flexible
• From Hrabik, Leahy, Minter, Westover

10
Types of Interviews

• Informal Conversation
• Guided Interview
• Structured Interview

Adapted from M. Patton, 1990 by Hrabik, Leahy,
Minter, Westover
11
A Nifty Form of Qualitative WorkAppreciative
Inquiry (Hammond, 1996)

• The key principle of AI (not artificial
  insemination, you aggies) is that it is a
  generative process
• Data are not always discovered and collected
• The data simply are it depends on your
  purpose and therefore may be generated or
  constructed by various means that fit your
  purpose

12
AI

• We are good at talking about what did not work
• Looking for what works worked
• What do we do well
• What has made us feel successful, effective,
  i.e., good about our work and roles

13
Assumptions of AI

• Every group does something well
• Our focus becomes our reality
• Asking questions influences the group
• Linking past successes to future events is
  powerful
• elements of social justice

14
AI Key Things GVs Take

• Lots of listening
• Throw out assumptions and try to check your ego
  at the door
• You are not neutral, so be sure to be a good
  recorder (i.e. listen and record w/o too much
  translation)
• You eventually will have the opportunity to
  dialog (discourse is the 50 term)
• It takes more time

15
AI is not Problem Solving

• PS A Problem to Solve
• ID problem
• Analysis of causes
• Analysis of possible solutions
• Action plan
• Action

• AI A mystery to be embraced
• Looking for the good
• Envision what might be
• Discourse on What might be
• Innovate What will be

Or is it?
16
The Power and the Jumble

• The power is when a group becomes engaged.
  Remember that listening part and the basic rule
  that people like to talk about themselves.
• At first, the data generatedi.e., from your
  field and interview notes may feel like nothing
  but a jumble of
• unorganized stuff

17
A case example
I thought I smelled something!
18
A Case Example

• A needs assessment, a program planning process,
  and an evaluation (sometimes it does not matter)
• What do you value most about 4-H as a parent?
• What are the benefits of 4-H for your kids?
• What would you hope to see Spokane County 4-H
  do in the future?

19
Qualitative Questions

• Well Suited
• Documenting and explaining real events
• Understanding how participants make sense and
  give meaning to their life
• Identifying unanticipated or taken-for-granted
  influences and phenomena

• Not As Well Suited
• Testing relationships or establishing cause and
  effect
• Addressing the generality or wider prevalence of
  the phenomena being studied or evaluated
• Comparing groups or a relationship among
  variables
• Predicting outcomes
• (Schram, 2003)

20
Good Questions Conducting Interviews/Focus G.s

• Keep your goals in mind.
• Specific the rationale for the discussion you
  have with the stakeholders or group
• Keep those questions open-ended
• Field test your questions
• Keep them direct and brief
• Consider the age group with whom you will be
  talking (field testing helps)

21
That you can dare to be different and still be
valid and reliable
22
The art of questioning
23
Wait Time M. Bud Rowe

• Waiting for what?
• Wait time I
• Wait time II
• Problems with I dont know (with kids)
• Will not occur as often
• Can also Pass
• Can also Ask another for help
• The posture and attitude of the facilitator is
  critical

24
Questioning Habits Penick, et al.

• H istory
• R elationships
• A pplication
• S peculation
• E xplanation
• ??Evaluation??

25
Things to Remember!

• Practice your wait time
• How can you do this?
• How will the student react
• Practice your listening skills - all the time!

26
Things to Remember!

• Ask one question at a time
• Avoid why questions whenever possible
• Only ask a question when you expect an answer
  (avoid rhetorical questions)

27
Well hey, these things just snap right off!
28
Garys Top 10 at this momentwords from the
not-too-wise

• Love the so what
• Short term, intermediate term, and long term
  outcomes are the Extension Holy Grail
• If you cant suspend or curb your opinions
  biasask for a fresh look from a respected
  colleague
• Formative is kuhl summative goes on a shelf
• Look for the surprises, it will make it more
  interesting

• If you think you have the definitive answer
  think again
• When you define your questions, attend only to
  what matters.
• Using a mixed methodology might be your answer
• You are not a purely objective observer (so get
  over it ?)
• Plan your evaluative inquiry as you plan your
  project and dont forget to share that so whatin
  writing!

29
(No Transcript)