An Introduction to Research

1
An Introduction to Research

• WESTOP Data Collection

2
Lee Fulmer

• WESTOP Research and Technology Committee

3
Agenda

• Why do research?
• What is research?
• Types of research
• What does DOE say about research?
• Exercise
• WESTOP data collection

4
Why do Research?

• To find out what is effective
• Accountability
• Mathematica

5
What is Research?

• Definitions
• Information gathering
• Collecting and analyzing new information in order
  to increase our understanding

6
What Research is

• Attempt to achieve systematically and with the
  support of data the answer to a question, the
  resolution to a problem, or the greater
  understanding of a phenomenon
• Generation of new information and testing of ideas

7
Characteristics of Formal Research

• Starts with a question or problem
• Requires a clear articulation of a goal
• Follows a specific plan or procedure
• Often divides the principal problem into more
  manageable subproblems

8
Characteristics of Research Contd

• 6. Specify assumptions youre making
• 7. Collection and interpretation of data

9
Types of Research

• A man from remote village is the first in his
  village to fly
• Friend 1 How far did you go? How high did you
  fly?
• Friend 2 How did you feel moving so fast? What
  was it like being above the clouds?

10
Types of Research

• Quantitative answer questions about data that
  can be measured in order to explain and predict
• Qualitative answer questions about nature of
  phenomena in order to describe phenomena and
  understand it from the participants point of view

11
What the Dept. of ED says

• No Child Left Behind
• Requires that teachers use techniques and methods
  that have been shown by rigorous research to be
  effective
• Randomized control trial

12
Randomized Control Trial

• Definition people randomly assigned to an
  intervention group or to a control group, in
  order to measure the effects of the intervention
• Unique feature Best chance of objectively
  demonstrating cause and effect
• Superior to other designs in measuring
  interventions effect

13
Potentially Misleading Designs

• Pre-post study Examine whether participants
  improve or regress during intervention
• Even Start
• Summer Training and Education Program

14
Potentially Misleading Designs

• Comparison GroupComparison group chosen by other
  than randomization
• Hormone replacement therapy
• Well-matched comparison groups can suggest
  trends, but gold standard is control group

15
Responses to DOE

• American Evaluation Association
• http//www.eval.org/doestatement.htm
• American Educational Research Association
• http//www.eval.org/doeaera.htm
• National Education Association
• http//www.eval.org/doepage.htm

16
Uses of WESTOP Data

• Illustrate program effectiveness
• Within your institution or agency
• To potential funders
• In grant proposals
• Research on program effectiveness

17
Data Collection

• Two stages
• Initially collect data on prior experience
• illustrate program effectiveness
• Eventually collect additional data that may be
  relevant for research

18
Why Prior Experience?

• Aggregate data
• Speaks to legislative intent

19
Why Two Stages?

• Prior experience data a modest data set and
  perhaps the most important
• Work out bugs
• Gauge participation level
• Second stage will require specifying which data
  to collect

20
Discussion

• Is the collection of regional prior experience
  data useful to you?
• What other types of regional data would you like
  to see collected?

21
References

• Identifying and Implementing Educational
  Practices Supported by Rigorous Evidence A User
  Friendly Guide. 2003. US. Dept of Education
  Institute of Education Sciences National Center
  for Education Evaluation and Regional Assistance.
  http//www.ed.gov/rschstat/research/pubs/rigorouse
  vid/index.html Downloaded 1/5/05.

22
References

• Practical Research Planning and Design. (1997)
  Leedy, Paul with contributions by Timothy J.
  Newby and Peggy A. Ertmer. Prentice Hall, Inc.
  Sixth Edition.