Exploration Projects within the Education Research Grants Program (84.305A) and Special Education Research Grants Program (84.324A)

1
Exploration Projects within the Education
Research Grants Program (84.305A) and Special
Education Research Grants Program (84.324A)

• Allen Ruby
• National Center for Education Research

2
Information for Applying

• Requests for Applications
• Letter of Intent
• IES Grants.gov Application Submission Guide
• Application Package

3
Requests for Applications (RFA)

• A separate RFA for each grant program (84.305A
  and 84.324A)
• Describes the requirements for an application
• Requests for Applications are available on
  http//ies.ed.gov/funding
• To be informed about the release of future RFAs,
  sign up for the IES Newsflash http//ies.ed.gov/n
  ewsflash/

4
Letter of Intent (LOI)

• A short description of your intended application
• PI, institution, collaborators
• Budget rough estimate
• Up to 1 page abstract describing the work
• Purpose
• Used by program officers to discuss your idea
• Used by IES Office of Standards and Review to
  plan for peer review process
• Not used in the peer review process superseded
  by your application
• Submitted on http//iesreview.ed.gov

5
Application Packages

• Contains the forms to be filled out and submitted
  as your application
• Available at www.grants.gov
• Help support_at_grants.gov or 1-800-518-4726
• For the June 23, 2011 application deadline,
  packages will be available starting April 21,
  2011
• For the September 22, 2011 deadline, packages
  will be available starting July 21, 2011
• Packages are specific for grant program and
  deadline

6
IES Grants.gov Application Submission Guide

• Instructions for completing and submitting the
  application package
• Available on http//ies.ed.gov/funding

7
Key Dates
Application Deadline Letter of Intent Due iesreview.ed.gov Application Package Available www.grants.gov Start Dates
6/23/11 4/21/11 4/21/11 3/1/12 to 9/1/12
9/22/11 7/21/11 7/21/11 7/1/12 to 9/1/12
8
Education Research Grants Program
(84.305A)Special Education Research Grants
Program (84.324A)
9
Grant Topics

• All applications to 84.305A and 84.324A must be
  directed to a specific topic
• Identify the appropriate topic for your work
• Read the Topic sections of the Request for
  Applications (RFA)
• Discuss with the appropriate program officer
  (listed in the RFA)

10
Education Research Topics (84.305A)

• Reading and Writing
• Mathematics and Science Education
• Cognition and Student Learning
• Social and Behavioral Context for Academic
  Learning
• Education Technology
• Effective Teachers and Effective Teaching
• Improving Education Systems Policies,
  Organization, Management, and Leadership
• Postsecondary and Adult Education
• Early Learning Programs and Policies
• English Learners

11
Special Education Research Topics (84.324A)

• Early Intervention and Early Learning in Special
  Education
• Reading, Writing, and Language Development
• Mathematics and Science Education
• Social and Behavioral Outcomes to Support
  Learning
• Transition Outcomes for Special Education
  Secondary Students
• Cognition and Student Learning in Special
  Education
• Professional Development for Teachers and Related
  Service Providers
• Special Education Policy, Finance, and Systems
• Autism Spectrum Disorders
• Technology for Special Education
• Families with Children with Disabilities

12
Grant Research Goals

• All applications to 84.305A and 84.324A must be
  directed to a specific research goal (1 of 5)
• The goal describes the type of research to be
  done
• As a result, every application is directed to a
  specific topic/goal combination
• Note topic and goal on your application (e.g., at
  top of abstract)

13
The 5 Research Goals

• Exploration
• The focus of this webinar
• Development and Innovation
• Develop a new or modify an existing intervention
• Efficacy and Replication
• Evaluate an intervention under ideal conditions
• Scale-up Evaluation
• Independently evaluate an intervention under
  routine conditions
• Measurement
• Develop and/or validate a measure

14
The Exploration Goal
15
Before Development Work Begins...

• Need to understand the problem
• Examine the underlying processes that affect
  education outcomes
• Look for malleable factors that might be targets
  for interventions
• Identify what distinguishes between effective and
  less effective practices

16
Before Evaluation Begins...

• Identify education interventions or components of
  them that appear promising due to their
  association with improved student outcomes before
  committing resources to evaluation

17
In a Nut Shell...

• Development work and some evaluations begin with
  Exploration Projects that examine underlying
  processes for the purpose of informing the
  development of new interventions, modifying
  existing interventions, or identifying promising
  interventions for evaluation

18
Exploration

• Explore the association between malleable factors
  and education outcomes
• A malleable factor can be changed by the
  education system be it a characteristic of
  students (e.g., skills, behaviors), teachers
  (e.g., credentials, practices) or school (e.g.,
  climate, size) , or an education program or
  policy
• Underlying processes that enhance or inhibit
  learning
• Aspects of a school, district, or community
  associated with beneficial education outcomes
• Education interventions associated with
  beneficial education outcomes (e.g., professional
  development, curricula, policies)
• Explore factors that mediate or moderate the
  relationship between malleable factors and
  student outcomes

19
Education Outcomes are for Students

• Preschool
• School readiness
• Developmental outcomes for infants and toddlers
  with disabilities
• Kindergarten through Grade 12
• Academic outcomes in reading, writing, math and
  science
• Behaviors, interactions, and social skills that
  support learning in school and successful
  transitions to post-school opportunities
• High school graduation
• Functional outcomes that improve educational
  results, transitions to employment, independent
  living, and postsecondary education for students
  with disabilities

20
Education Outcomes (continued)

• Postsecondary
• Access, persistence, completion
• Achievement in gateway math science courses
• Achievement in introductory composition courses
• Adult Education
• Reading, writing, and math for basic and
  secondary education and English Language Learners

21
Exploration Projects Should

• Identify underlying processes that enhance or
  inhibit learning. These may contribute to the
  development or modification of interventions
• Identify education interventions that may deserve
  rigorous evaluations
• Identify factors that mediate or moderate the
  underlying processes or the interventions
• Generate hypothesis concerning causal relations
  between factors and outcomes
• Contribute to theories of action

22
Exploration Projects Should NOT

• Test the efficacy of education interventions
• Examine non-malleable factors
• Examine malleable factors outside the control of
  the school system

23
The Applications Research Narrative

• Key part of your application
• 4 Sections
• Significance
• Research Plan
• Personnel
• Resources
• Each section scored and an overall score given
• Requirements vary by topic research goal
• 25 pages, single spaced

24
Significance Section

• Describes the overall project
• The research question to be answered
• The malleable factors, moderators, and mediators
  to be examined
• Rationale for the work
• Theoretical justification
• Logic Models, Change Models
• Empirical justification
• Related work
• Practical justification
• importance of the variables (malleable factors,
  mediators, moderators, outcomes)

25
Significance Section

• How work will lead to useful next step
• Development or modification of interventions to
  address the identified malleable factors or
  underlying process to improve student outcomes
• Identification of interventions for more rigorous
  evaluation
• Overall importance

26
Significance 2 Problems Often Seen in
Applications

• Unclear theory of change
• Why should the malleable factor be related to
  improved outcomes
• A well laid out theory of change makes clear what
  is expected to happen and in what order
• Easy for reviewers to understand research plan
  why measure certain outcomes
• Graphic can be helpful e.g. a logic or change
  model

27
Significance 2 Problem Areas

• Unclear Description of Intervention
• What the intervention is
• Many components and may be applied at different
  times how fit together Graphic may help
• Intervention not shown to be strong enough to
  expect an impact
• Informational
• Ensure fidelity
• Overly focused on actions not content
• Ex. 20 hours of PD held over 10 weeks but no
  detail on what is to be covered in the sessions

28
Research Plan

• Describe the work you intend to do
• How you will answer your research question
• Make certain the Research Plan is aligned to
  Significance section
• All research questions should have justification
  in Significance.
• Step-by-step process
• Timeline to show when everything will be done

29
All Research Plans Should Include

• Clear, concise hypotheses or research questions
• Well-specified relations between hypotheses,
  measures, and independent dependent variables
• Clear description of data sources
• Detailed descriptions of data analysis procedures

30
The Research Plan Should Describe

• Setting
• Population and sample
• Sampling plan inclusion and exclusion criteria
• Size (power issue) and attrition
• External validity
• Measures
• Outcomes
• Proximal and distal
• Sensitivity
• Other measures
• Reliability and validity

31
Work May Include

• Original data collection with appropriate
  statistical analyses
• Secondary data analysis of existing datasets
• Secondary data analysis complemented by primary
  data collection
• Meta-analyses designed to determine moderators or
  moderators of effects

32
Primary Data Collection

• Sampling strategy
• Sample characteristics
• Variables to be measured
• Procedures for data collection
• Procedures for coding data
• Describe links to secondary data if relevant

33
If observational data are collected

• How inter-observer reliability maintained
• How data be coded
• How will data be quantified to support prediction
  of relation between what was observed and
  outcomes of interest

34
Secondary Data Use

• Database(s)
• Sample characteristics
• Variables to be used
• Access to and permission to use the data
• Linking if multiple datasets used
• Malleable factor
• Outcome variables
• Mediators moderators

35
Exploration Work is Non-Causal

• Descriptive analysis
• Statistical correlational analysis
• Can include more complex methods to address
  selection issues
• Mediation analysis

36
Meta-analysis

• Criteria for including or excluding studies, and
  rationale
• Search procedures used
• Coding scheme procedures for extracting data
  from studies
• Procedures for ensuring reliability of the coding

37
Meta-analysis

• Demonstrate that sufficient numbers of studies
  are available to support meta-analysis
• Demonstrate that relevant information is reported
  frequently enough that a database can be
  constructed
• Clearly describe effect size statistics,
  associated weighting functions, procedures for
  handling outliers, and any other adjustments

38
Research Plan Analysis

• Describe how analysis answers research questions
• Show your models
• Address clustering
• Describe how missing data will be handled
• Check for equivalency of groups at start of study
• Track attrition and check for attrition bias
• Describe sensitivity tests of assumptions
• Describe analysis of qualitative data and links
  to quantitative analysis

39
Personnel Section

• Describe key personnel
• Link each person and their expertise to their
  role in project - show that every aspect of
  project has person with expertise to do it
• Methodologists show expertise in particular
  method to be used
• Substantive person for all issues addressed
• Do not propose to identify and hire key people
  after grant awarded
• Project management skills
• Give time contribution for each - show that every
  aspect has enough time from expert
• Orient CVs same way specific to project
• 4 pages plus 1 page for other sources of support

40
Personnel Requirements

• Publication record and projected publications
  from this grant are considered
• Developers should discuss past success getting
  developed interventions evaluated
• If previous IES grant, discuss results
• Evaluations require attention to objectivity
  should a developer or persons with financial
  interest be involved
• Efficacy projects address how objectivity
  maintained
• Scale-Up Independent evaluation developer can
  provide routine implementation support

41
Personnel Strategies for PI

• Senior Researcher
• Show adequate time to be PI
• Make credentials clear not all reviewers may
  know
• Junior Researcher as PI
• Show adequate expertise not only to do work but
  to manage project
• Reviewers may be more comfortable if you have
  senior person(s) on project to turn to for advice

42
Resources

• Show the institutions involved have the capacity
  to support the work
• Do not use university boilerplate
• Show that all organizations involved understand
  and agree to their roles
• What will each institution, including schools,
  contribute to the project
• Show strong commitment of schools and districts
• Have alternatives in case of attrition

43
Resources (continued)

• Appendix C should back this up with
• Detailed Letters of Support from research
  institutions, States, districts, schools
• Data issues
• Document permission to use and access to
  confidential data (letters in Appendix C)
• Show familiarity with data show that it can be
  used to do the proposed work
• If merging datasets, show that it can be done

44
Appendices

• Appendix A (15 page limit)
• Figures, charts, and tables
• Examples of measures
• 3 pages to address past reviewer comments or to
  argue that a proposal is a new submission
• Appendix B (10 page limit)
• Examples of materials used in an intervention or
  assessment
• Appendix C (no page limit)
• Letters of agreement (districts, schools, data
  providers, other partners, consultants)
• Clearly state responsibilities of the writer

45
Budget and Budget Narrative

• Provide a clear budget and budget narrative for
  overall project and each sub-award
• IES Grants.gov Application Submission Guide
  describes budget categories
• Check RFA for specific budget requirements for
  Research Goals and Grant Programs
• Ensure agreement among Research Narrative,
  Budget, and Budget Narrative

46
Award Information

• Secondary data analysis or meta-analysis
• Typical 100,000 to 300,000 per year (direct
  and indirect)
• Maximum 2 years and 700,000
• Primary data collection and analysis (with or
  without a secondary data analysis)
• Typical 100,000 to 400,000 per year
• Maximum 4 years and 1,600,000

47
Grant Submission

• Make sure your institution is registered on
  grants.gov
• Complete your online forms and upload PDFs
• Authorized representative completes the process
• Submit by 43000 EST on deadline earlier is
  safer
• If problems uploading
• Contact Help Line 1-800-518-4726 and get a case
  number
• You should receive four emails
• Grants.gov assigns you a number that starts with
  GRANT
• Grants.gov your application is validated or
  rejected due to errors. If the latter, correct
  and resubmit until validated.
• Dept. of Ed retrieved your application from
  Grants.gov
• Dept. of ED assigns you a number that starts
  with R305 or R324

48
Application Review)

• Carried out by the IES Office of Standards and
  Review
• Compliance screening for format requirements
• Responsiveness screening to topic and goal
  requirements
• Assigned to review panel
• 2-3 reviewers (substantive and methodology)
• If scored high enough, application is reviewed by
  full panel
• Many panelists will be generalists to your topic
• There will an expert in every procedure you use
• Overall score plus scores on Significance,
  Research Plan, Personnel, and Resources
• Resubmissions encouraged address comments

49
Peer Review Process Information

• http//ies.ed.gov/director/sro/peer_review/index.a
  sp

50
Notification

• All applicants will receive e-mail notification
  of the status of their application
• All applicants receive copies of reviewer
  comments
• If you are not granted an award the first time,
  plan on resubmitting and talk to your program
  officer

51
Final Reminders

• Start early
• Read the Request for Applications
• Talk with the program officer
• Start the online submission process early

52
ies.ed.gov

• Allen Ruby
• allen.ruby_at_ed.gov