IES Grant Writing Workshop for Efficacy and Replication Projects
Joan E. McLaughlin, Ph.D. Deputy Commissioner National Center for Special Education Research
Overview of Presentation • • • • • • •
Structure of the Institute of Education Sciences Research Topics within NCSER & NCER Goals within Research Topics Research Narrative for Goal 3 applications Preparing and Submitting an Application What Happens Next? Questions
Structure of the Institute of Education Sciences
IES Organizational Structure Office of the Director
National Center for Education Research
National Center for Education Statistics
National Board for Education Sciences
National Center for Education Evaluation
National Center for Special Education Research
FY 2012 Research Grant Topics in NCSER and NCER
Special Education Research Topics (84.324A) • • • • • • • • • • •
Early Intervention and Early Learning in Special Education Reading, Writing, and Language Development Mathematics and Science Education Social and Behavioral Outcomes to Support Learning Transition Outcomes for Special Education Secondary Students Cognition and Student Learning in Special Education Professional Development for Teachers and Related Service Providers Special Education Policy, Finance, and Systems Autism Spectrum Disorders Technology for Special Education Families with Children with Disabilities
Education Research Topics (84.305A) • • • • • • •
Reading and Writing Mathematics and Science Education Cognition and Student Learning Social and Behavioral Context for Academic Learning Education Technology Effective Teachers and Effective Teaching Improving Education Systems: Policies, Organization, Management, and Leadership • Postsecondary and Adult Education • Early Learning Programs and Policies • English Learners
Research Goals within Topics
Solution Driven Research Research intended to contribute to the solution of practical education problems: • • • • •
Exploration Development and Innovation Efficacy and Replication Scale-up Measurement
Goal One Goal Two Goal Three Goal Four Goal Five
Exploration • Explore the association between malleable factors and education outcomes – A malleable factor can be changed by the education system be it a characteristic of students (e.g., skills, behaviors), teachers (e.g., credentials, practices) or school (e.g., climate, size), or an education program or policy • Underlying processes that enhance or inhibit learning • Aspects of a school, district, or community associated with beneficial education outcomes • Education interventions associated with beneficial education outcomes (e.g., professional development, curricula, policies)
• Explore factors that mediate or moderate the relationship between malleable factors and student outcomes
Development and Innovation • Develop new interventions (e.g., instructional practices, curricula, teacher professional development) • Demonstrate the feasibility of the intervention for implementation in an authentic education delivery setting • Collect pilot data on promise of intervention to achieve intended outcomes
Efficacy and Replication • Causal test of whether or not a fully developed intervention has a beneficial impact on student outcomes relative to a counterfactual in an authentic educational setting • Takes place under “ideal” conditions – Homogenous sample of students/schools – Extra assistance to support high implementation fidelity
Scale-up Evaluation • Independent causal test of whether or not a fully developed efficacious intervention has a beneficial impact on student outcomes relative to a counterfactual in an authentic educational setting under routine implementation – Independent: evaluation team has no financial interest in intervention – Efficacious: evidence of intervention’s efficacy – Routine implementation: as implemented by practitioners with expected level of support if adopted by a school or district
Goal 5: Measurement • Develop and validate assessments or other measurement tools
Overview of Efficacy and Replication Grant Applications
Efficacy • Efficacy is the degree to which an intervention has a net positive impact on the outcomes of interest relative to the program/practice to which it is being compared
Replication • Replication projects determine if an intervention that has produced a positive impact under one set of conditions will produce a positive impact under different conditions (e.g., the same intervention with different populations of students, different educational settings, or different levels of support)
Purpose of Efficacy and Replication Projects • To estimate the strength or potency of the impact of the intervention • To inform the degree to which the intervention can be feasibly or practically implemented • To assess implementation fidelity
Appropriate Interventions for Efficacy and Replication Projects • Interventions that are widely used but have not been rigorously tested • Interventions that are fully developed, have evidence of their feasibility and promise, but are not widely used • Interventions that have a positive effect under one set of conditions
Efficacy and Replication Projects • Use experimental or quasi-experimental research designs that minimize selection bias or allow it to be modeled • Employ a limited set of conditions • Often involve more support from the intervention developer/researcher than is typically available
What percentage of research grants are Efficacy and Replication projects? • 22 to 23 percent of the research projects funded by IES are Efficacy and Replication projects
Efficacy and Replication Applications Research Narrative
Components of the Research Narrative • • • •
Significance Research Plan Personnel Resources
Significance • Describe fully-developed intervention • Provide rationale for why testing the efficacy of this intervention is important • Present a clear theory of change to guide the understanding of the intervention and expected outcomes
Research Plan (1) • Clearly articulate all research questions – Does the intervention lead to a meaningful improvement in education outcomes? – What are the conditions that support or hinder the implementation of the intervention? – What processes mediate the relation between the intervention and student outcomes?
Research Plan (2) • Define sample (include size requirements, exclusion rules, and discussion of attrition issues)
Research Plan (3) • Research Design – Use a randomized controlled trial (RCT) whenever possible – Clearly justify the use a quasi-experimental design and address internal validity threats
Research Plan (4) • Describe the power analysis - Provide rationale for assumptions - If subgroups are integral to your study, consider whether you have sufficient power to address the relevant research questions
Research Plan (5) • Provide detailed description of the chosen measures (include information on appropriateness, reliability, validity) – Don’t forget to detail any researcher-developed measures
• Provide clear description of data collection procedures and timing of data collections
Research Plan (6) • Describe how fidelity will be assessed • Clearly describe the comparison condition • Identify mediating and moderating variables and how they will be collected • Detail the plan for analysis of qualitative and quantitative data
Personnel • • • •
Content domain experts Methodological expertise Statistical expert Staff experienced in conducting research in chosen setting and with population
Personnel (2) • • • •
Qualifications Roles and Responsibilities Percent of time devoted to the project DON’T FORGET to address the issue of the objectivity of the evaluation if the intervention developer is part of the team
Resources • Address institutional resources • Document the support of those in the research setting (e.g., letters from school districts or schools, preschools)
Efficacy and Replication • Typical award: $250,000 to $650,000 per year • Maximum of 4 years and $3,500,000
Efficacy Follow-up Studies • Examine the sustainability of the impacts of an intervention after the original study on: - students that received the intervention and have moved on - a new group of students in the same setting when additional resources are no longer provided for the intervention
Efficacy Follow-up Studies • Emphasis in narrative on previous study, including sample and effects • Typical award: 150,000 to $300,000 per year • Maximum of 3 years and $1,200,000
Preparing and Submitting an Application
Getting Started q Request for Applications q IES Grants.gov Application Submission Guide q Application Package
Relevant Requests for Applications (RFAs) • Education Research Grant Program (84.305A) • Special Education Research Grant Program (84.324A)
Documents on the IES Website FY 2012 Requests for Applications and the Grant Submission Guide are available on: http://ies.ed.gov/funding Sign up for the IES Newsflash: http://ies.ed.gov/newsflash/
http://ies.ed.gov
Finding Application Packages • FY 2012 Application Packages are available on www.grants.gov • June Application Package available on April 21, 2011 • September Application Package available on July 21, 2011
Letters of Intent • • • • • •
For June competition, due 4/21/2011 For September competition, due 7/21/2011 Non-binding, but strongly encouraged Submitted electronically Instructions at iesreview.ed.gov Receipt via e-mail
IES assistance • Read the Request for Applications carefully. • Call or email IES program officers early in the process. • As they have time, IES program staff will review draft proposals and provide feedback to applicants.
Follow Directions and Attend to Details Read Program Announcements and Application Instructions carefully • • • • •
Eligible Applicants Special Requirements Content and Formatting Requirements Application Submission and Processing Application Due Dates
Application Due Dates in 2011 • Education and Special Education Research:
June 23 and September 22 At 4:30:00 p.m. (EASTERN)
What Happens Next?
Peer Review • Proposal is reviewed for compliance. • Compliant proposals are assigned to a review panel. • For Goal 3, three panel members conduct primary review of each application. • Most competitive applications are reviewed by full panel at the panel meeting.
Notification • All applicants will receive email notification of the status of their application. • All applicants receive copies of reviewer comments. • Applicants who do not receive an award should consider revising and resubmitting their application. Applicants are encouraged to contact the program officer about the resubmission.
Questions
Frequently Asked Questions • Advantage of June vs September submission? No. • Possible start dates? For June submission: 3/1/12 – 9/1/12 For September submission: 7/1/12- 9/1/12
Frequently Asked Questions (2) • Can I plan to resubmit in the next competition if I don’t get funded? There is generally not enough time for you to get the reviews, attend to the comments and resubmit. • How long does the review process take? About 8 months.
Special Education Research queries: Joan McLaughlin
[email protected]
Education Research queries: Elizabeth Albro
[email protected]
