Evaluation Research

1
Evaluation Research

• Methodology Presentation
• March 27, 2007
• LIS5271

Team 9 Jenniferanne Broido Julie Camp Lisa
Duque Tina Smith Ryan Valentin
2
What is Evaluation Research?
3
History
4
Common Types

• Experimental Designs
• This involves use of traditional experiment
  techniques explored in chapter 8 of Babbies
  textbook. As Babbie says, At base, experiments
  involve (1) taking action and (2) observing the
  consequences of that action (Babbie, 2006, 221).
  In the context of evaluation research, one would
  design an experiment to test a new initiative and
  then evaluate the initiatives effectiveness.

• Quasi-Experimental Designs
• This design is very similar to an experimental
  one, except participants are not randomly divided
  into control and experimental groups. This is
  because In evaluation research, its often
  impossible to achieve such an assignment of
  subjects (Babbie, 2006, 357).Within the
  category of quasi-experimental designs, there are
  three specific types
• Time-Series Designs
• Non-equivalent control groups
• Multiple time-series designs

Qualitative Designs This design type utilizes
basic qualitative research methods, such as
in-depth interviews. However, Babbie suggests
using both qualitative and quantitative methods
in evaluation research. As Babbie says, Making
statistical comparisons is useful, and so is
gaining an in-depth understanding of the
processes producing the observed results or
preventing the expected results from appearing
(Babbie, 2006, 362).
5
Key Characteristics

• Evaluation is a form of applied research
• that is, its intended to have some real-world
  effect
• Many Methods
• surveys, experiments, etc, can be used in
  evaluation methods (349)
• It refers to a RESEARCH PURPOSE rather than a
  specific method
• (Babbie, 2006, 349)
• The research purpose is to evaluate the impact of
  social interventions such as
• new teaching methods
• evaluating whether drivers education courses
  prevent or reduce traffic accidents
• the impact of rape reform legislation
• effectiveness of wife battering prevention
  policies
• the negative social consequences of pornography

6
Strengths

• Evaluation research offers many strengths and
  potential contributions toward the betterment of
  human life.
• Evaluation research leads to a better
  understanding of whether a program is effective,
  whether a policy is having the desired impact,
  whether a chosen approach is the most
  cost-effective way of accomplishing agreed-upon
  goals, how an effort can be improved, and whether
  an initiative should be continued (WestEd, 2007).
• Evaluation research helps to judge whether social
  programs have succeeded or failed.
• Evaluation research also encourages use of
  program evaluation and helps to justify decisions
  about implementing needed programs.

7
Weaknesses

• Evaluation research takes place within our
  uncontrollable daily lives, and getting subjects
  to do what theyre supposed to do is often a
  challenging task. This lack of control of the
  subject can create real dilemmas for the
  researcher (Babbie, 2006, 363-64).
• Another logistical problem is administrative
  control causing the experimental stimulus to
  change in the middle of the research project.
  Some of the data will reflect the original
  stimulus other data will reflect the
  modification.
• Social interventions will throw the researchers
  directly into political, ideological and ethical
  issues and it may be difficult to remain
  impartial.
• The results are often argued about. Studies
  providing clear results and obvious policy
  implications are ignored if they sometimes
  contradict political or deeply held beliefs
  (Babbie, 2006, 365).Evaluation research leads to
  a better understanding of whether a program is
  effective, whether a policy is having the desired
  impact, whether a chosen approach is the most
  cost-effective way of accomplishing agreed-upon
  goals, how an effort can be improved, and whether
  an initiative should be continued (WestEd, 2007).

8
Issues To Consider
9
Article 1

• Fagan, Jody Condit and Margie Ruppel. (2002).
  Instant messaging reference users evaluation of
  library chat. Reference Services Review, 303,
  pp. 183-197. Retrieved March 22, 2007, from FSUs
  Emerald Insight database.

Could instant messaging be used for providing
reference services?
10
Article 2

• Hersh, William R., Katherine Junium, Mark
  Mailhot, and Patricia Tidmarsh. (2001).
  Implementation and Evaluation of a Medical
  Informatics Distance Education Program. Journal
  of the American Medical Informatics Association,
  86, 570-584. Retrieved March 22, 2007, from
  FSUs Highwire Press American Medical Informatics
  Association database.

What content and type of distance-learning
program would be of interest for continuing
education in medical informatics?
11
Sources Cited

• Babbie, Earl. (2006). The Practice of Social
  Research (11th Ed.). New York, NY Wadsworth
  Publishing Co.
• Fagan, Jody Condit and Margie Ruppel. (2002).
  Instant messaging reference users evaluation of
  library chat. Reference Services Review, 303,
  pp. 183-197. Retrieved March 22, 2007, from FSUs
  Emerald Insight database.
• Garson, G. David. (n.d.). Evaluation research
  topic from StatNotes - the OnlineTextbook for PA
  765. Retrieved March 24, 2007, from
  http//www2.chass.ncsu.edu/garson/pa765/evaluation
  .htm
• Hersh, William R., Katherine Junium, Mark
  Mailhot, and Patricia Tidmarsh. (2001).
  Implementation and Evaluation of a Medical
  Informatics Distance Education Program. Journal
  of the American Medical Informatics Association,
  86, 570-584. Retrieved March 22, 2007, from
  FSUs Highwire Press American Medical Informatics
  Association database.
• WestEd. (2007). Evaluation Research. Retrieved
  March 24, 2007, from http//www.wested.org/cs/we/
  view/pg/4