Educational Research 102: Selecting the Best Study Design for your Research Question

1
Educational Research 102Selecting the Best
Study Design for your Research Question

• Francis S. Nuthalapaty, MD

2010 APGO Faculty Development Seminar
2
Disclosures

• No relevant financial disclosures to declare

3
Learning Objectives

• Describe types of research and study designs
• Understand the characteristics of a good research
  question
• Understand how to convert the research question
  into a hypothesis
• Understand how to select the most appropriate
  study design to test the hypothesis

4
What do we already know?
Educational Concept True or False?
1. High-fidelity medical simulations facilitate learning.

2. Restriction in resident duty hours improves the quality of patient care.

TRUE
FALSE
1. Issenberg SB, McGaghie WC, Petrusa ER, Gordon
DL and Scalese RJ (2005) Features and uses of
high-fidelity medical simulations that lead to
effective learning a BEME systematic review.
Medical Teacher 27, 2, pp 10-28. 2. Fletcher KE,
Davis SQ, Underwood W, Mangrulkar RS, McMahon LF
Jr, Saint S. Systematic review effects of
resident work hours on patient safety. Ann Intern
Med. 2004 Dec 7141(11)851-7.
5

• Write an educational research question in which
  you have interest

6
Types of Research

• Empirical vs. Non-empirical
• Basic vs. Applied

7
Empirical vs. Non-Empirical Research

• Empirical
• Involves collection of data first hand

• Non-Empirical
• No first hand data collection

8
Example Empirical or Non-empirical?

• The Effect of House Staff Working Hours on the
  Quality of Obstetric and Gynecologic Care
• Bailit, J et al, Obstet Gynecol 2009
• OBJECTIVE To measure the effect of house staff
  working hours reforms on the quality of obstetric
  and gynecologic care.

9
Example Empirical or Non-Empirical?

• Features and uses of high-fidelity medical
  simulations that lead to effective learning a
  BEME systematic review.
• Issenberg SB et al, Med Teach 2005
• OBJECTIVE Review and synthesize existing
  evidence in educational science that addresses
  the question, 'What are the features and uses of
  high-fidelity medical simulations that lead to
  most effective learning?'.

10
Types of Research

• Empirical vs. Non-empirical
• Basic vs. Applied

11
Basic vs. Applied Research

• Basic
• Results apply to a great many people and
  situations
• Results are related to general theory or to a
  general field of knowledge
• Results need not have immediate or even clear
  implications

• Applied
• Results are applicable only to a specific group
  of people in a particular situation.
• Results are not necessarily related to a broader
  field of knowledge
• Results must have immediate and clear
  implications for practice

12
Example Basic or Applied?

• Assessing Vaginal Surgical Skills Using Video
  Motion Analysis
• Diwadkar G et al, Obstet Gynecol 2009
• OBJECTIVE To demonstrate the feasibility of
  using video motion analysis to quantitate a key
  step of vaginal hysterectomy and define
  measurable differences between novice and
  experienced surgical trainees during vaginal
  hysterectomy.

13
Example Basic or Applied?

• Effects of a Depression Education Program on
  Residents Knowledge, Attitudes, and Clinical
  Skills
• Learman L et al, Obstet Gynecol 2003
• OBJECTIVE To determine whether an interactive
  educational program would improve obstetrics and
  gynecology Residents' knowledge, attitudes,
  confidence, and skills in caring for depressed
  patients.

14
Research Methods

• What is the difference between Qualitative
  Research and Quantitative Research?

15
Qualitative vs. Quantitative Research

• Qualitative
• Descriptive and exploratory focus
• Used to gain insight into attitudes, behaviors,
  values
• Analysis of unstructured information
• Narrative reporting

• Quantitative
• Hypothesis driven
• Used to identify association and/or causation
• Analysis of discrete variables
• Statistical reporting

16
Quantitative Research

• Experimental Quasi-experimental
• Non-Experimental
• Causal-comparative
• Correlational

17
Quantitative Research

• Experimental Quasi-experimental
• Assess effect of an independent variable on
  dependent variables
• Comparison of 2 or more groups
• Control over treatment measurement
• Randomization
• Control group

18
Quantitative Research Designs
Experimental
Treatment
Measurement
Study Population
Randomize
Control
Measurement
Post-test only Control Group
19
Quantitative Research Designs
Experimental
Treatment
Measurement
Measurement
Study Population
Randomize
Control
Measurement
Measurement
Pre-test/Post-test Control Group
20
Quantitative Research Designs
Experimental
Treatment
Measurement
Measurement
Control
Measurement
Measurement
Study Population
Treatment
Measurement
Randomize
Control
Measurement
Solomon Four Group
21
Quantitative Research Designs
Quasi-Experimental
Study Population
Treatment
Measurement
One-shot Case Study
22
Quantitative Research Designs
Quasi-Experimental
Study Population
Treatment
Measurement
Measurement
One-group Pre-test/Post-test
23
Quantitative Research Designs
Quasi-Experimental
Study Group 1
Treatment 1
Measurement
Study Group 2
Treatment 2
Measurement
Static Group
24
Quantitative Research Designs
Quasi-Experimental
Study Group 1
Measurement
Treatment 1
Measurement
Matching
Study Group 2
Measurement
Treatment 2
Measurement
Pre-test/Post-test Matched Subjects
25
Quantitative Research

• Non-Experimental
• Causal-comparative
• Correlational

26
Causal-Comparative

• Baseline differences are pre-existing
• Non-permutable Ethnicity, Gender
• Permutable Teaching style
• Determine the cause or consequences of
  differences
• Associations can be identified
• Causality cannot be determined

27
Quantitative Research Designs
Causal-Comparative
Exposures
Outcome
Control
Outcome
Exploration of Effects
28
Quantitative Research Designs
Causal-Comparative
Exposures
Outcome
Exposures
Control
Exploration of Causes
29
Quantitative Research Designs
Causal-Comparative
Exposure
Outcome
Group 1
Outcome
Group 2
Exploration of Consequences
30
Correlational Research

• Single group of subjects
• Describe degree to which 2 or more quantitative
  variables are related
• Help explain important human behaviors
• Predict likely outcomes
• Identifies associations
• Causality cannot be determined

31
Correlational Research
Subjects Observations O1 O2 Observations O1 O2
Student A
Student B
Student C
Student D
Student E
32
Correlational Research

• Correlation Coefficient
• Calculated by regression
• lt .35 only a slight relationship
• .40 - .60 possible theoretical value
• .50 minimum for crude predictions
• gt.65 reasonably accurate predictions
• gt.85 close relationship
• Positive correlation direct association
• Negative correlation inverse association

33
Threats to Internal Validity

• Subject characteristics
• Loss of subjects
• Maturation
• Repeated measures
• Statistical regression
• Investigator bias

34
External Validity

• Can the findings from the study be generalized to
  larger populations?
• Subject characteristics
• Environment (lab vs. natural setting)
• Did the subjects act differently b/c they were
  enrolled in a study (Hawthorne effect)?

35

• What are the steps in the research process?

36
The Research Process
Define Research Question
Hole in the literature
No hole in the literature
Conduct Literature Review
Refine Research Question
Obtain IRB Approval
Design Study
Collect Analyze Data
Disseminate Results
37
FINER Research Questions

• F Feasible
• I Interesting
• N Novel
• E Ethical
• R - Relevant

38
FINER Research Questions

• Feasible
• Can be investigated with available
• Time
• Money
• Space
• Other resources

39
FINER Research Questions

• Interesting
• Interesting to you
• Interesting to others in the field
• Novel
• Will answers to the question advance knowledge in
  the field?

40
FINER Research Questions

• Ethical
• Will any physical or psychological harm come to
  anyone as a result of the research?
• Protect participants from harm
• Ensure confidentiality of data
• Avoid knowing deception of participants
• Relevant
• How might answers to this question improve
  educational practice?

41

• What is a research hypothesis and how is it
  different from a research question?

42
Research Hypothesis

• A prediction regarding
• the possible outcomes
• of the study

Wallen Fraenkel. Educational Research A Guide
to the Process. 2nd Ed.
43
Research Hypothesis

• A statement which
• Summarizes the elements of the study
• The sample
• The design
• The predictor and outcome variables

Yoder E, MERC Workshop, 2009 AAMC Annual Meeting
44
Research Hypothesis

• Based upon a FINER research question
• Simple one difference or relationship
• Specific clearly stated, defined
• Stated in advance before data collection

Yoder E, MERC Workshop, 2009 AAMC Annual Meeting
45
Research Hypothesis

• May be stated
• As the alternative hypothesis
• Non-directional
• The difference is unknown
• Directional hypothesis
• Existing studies/data suggest direction of
  relationships
• As the null hypothesis

Yoder E, MERC Workshop, 2009 AAMC Annual Meeting
46
Research Hypothesis

• Ha There is a difference between groups or
  variables
• Female medical students have a different IQ than
  male medical students
• µf ? µm or µf gt µm
• H0 There is no difference between groups or
  variables
• Female medical students have a different IQ than
  male medical students
• µf µm

Yoder E, MERC Workshop, 2009 AAMC Annual Meeting
47
Example Hypothesis

• Improving Resident Competency in the Management
  of Shoulder Dystocia With Simulation Training
• Deering S, et al, Obstet Gynecol 2004
• METHODS Residents from 2 training programs were
  randomized by year-group to a training session on
  shoulder dystocia (SD) management that used an
  obstetric birthing simulator or to a control
  group with no specific training. Both groups were
  subsequently tested on a standardized SD
  scenario, and a physician grader rated the
  resident's performance with a standardized
  evaluation sheet.

48
Example Hypothesis

• What is the research question?
• What is the null hypothesis?
• What is the research study design?
• Basic or Applied?
• Qualitative or Quantitative?
• Experimental or Non-Experimental

49
Quantitative Research Designs
Experimental
Treatment
Measurement
Study Population
Randomize
Control
Measurement
Post-test only Control Group
50
Small Group Activity

• Share your research questions
• Select one research question to convert to a
  hypothesis
• Select the most appropriate study design

51
Give Your Research Impact

• Investigate important questions
• Connect your study to prior studies and help
  build the body of work
• Uses appropriate research design
• Consult with a biostatistician in the design
  phase
• Dissemenate your results

52
(No Transcript)
53

• Session content, including narrated MS Powerpoint
  slides available at
• http//www.obgynknowledgebank.net