Scientific%20Research%20in%20Education

1
Scientific Research in Education
NRC Committee on Scientific Principles in
Education Research

• Editors
• Richard J. Shavelson
• Lisa Towne

2
Why Are We Here?

• Castle bill, fall 2000
• Office of Educational Research and Improvement
  re-authorization
• OERI disbanded in 2002
• Education Sciences Reform Act
• Now Institute of Education Sciences
• Encourage self-reflection

3
Is That All?

• Thought about a federal agency
• Did NOT evaluate current research or techniques

4
Scientific MethodGuiding Principles

• Pose significant questions -- empirical
  investigation
• Link research to relevant theory
• Use methods w/ direct investigation
• Chain of reasoning
• Replicate and generalize
• Professional scrutiny

5
Properties of Education

• Values and Politics
• Differing views on human potential
• No single goal to current research
• Personal values too involved
• Decide on one goal, not best or final goal
• Governmental changes

6
Properties of Education

• Human Volition
• Individual desires confound experiments
• Parents are whiny and hate science
• Think of the children
• Think of them!
• Mobility of U.S. population
• 10/year ?

7
Properties of Education

• Variability of Education Programs
• Tinkering towards Utopia-Tyack Cuban 95
• State standards
• Organization
• Group-Class-Grade-School-District-County-State-Fed
  eral
• Cultural Diversity

8
Properties of Education Research

• Multidisciplinary
• Integrate/correlate from different fields
• Advances depend on advances in other fields
• Development of training programs is hard

9
Properties of Education Research

• Ethical Considerations
• Cannot create education-free control
• Cannot hide purpose (for double-blind)
• Somebody think of the children!
• Reliance on Practitioners
• Recent shift to long-term partnered research

10
Designing Research

• Question dictates design
• Three important facets
• Description what is happening?
• Cause is there a systematic effect?
• Mechanism why/how is it happening?

11
Description

• Collect raw valid data
• Random sampling (surveys and tests)
• Reproducible valid results
• In-depth case studies
• 1970s 23 college women
• Career in major, career outside major, housefrau
• Schoolwork predictor

12
Causation

• Find causal relationships between variables
• Isolate variables on random surveys/tests
• Non-random tests may generalize better
• Real-world education not dictated randomly
• Harder to validate - selectivity bias
• Model-fitting
• Salary vs. dropouts

13
Mechanism

• Bring all techniques to bear on how
• 10-year study on Catholic schools
• Three models
• Sector (private spiritual nature)
• Compositional (composition of student body)
• School (feature of school operation)
• Conclusion coherence (sense of community)

14
Mechanism When Theory is Weak

• College women study
• Why did nontraditional majors not have careers?
• Design study of ratio reasoning
• Real problems (ramps, exchange rates)
• Videotape of classroom activities
• Compared to other classrooms over 3 years
• Produced refined theoretical model

15
Designing An Ideal Federal Agency
An Exercise in Publicly Funded Common Sense

• Staff with Scientists
• Structures to Guide Agenda
• Insulate from Politics
• Balanced Research Portfolio
• Adequate, Consistent Funding
• Invest in Private Research Infrastructure