Connecting Research, Policy and Practice
Research Networks Focused on Cri4cal Problems of Educa4on Policy and Prac4ce Thomas Brock, Commissioner James Benson & Wai Chow Program Officers Na:onal Center for Educa:on Research ies.ed.gov
Presenta:on Outline I: II: III: IV: V: VI:
Background and Overview College Comple:on Science Teaching Network Lead Applica:on Requirements Preparing a Responsive, High Quality Applica:on
ies.ed.gov
2
SECTION I: BACKGROUND AND OVERVIEW • Networks grew out of year-‐long effort to solicit stakeholder input on our research programs and ways to improve – Technical Working Group mee:ngs – Public comment – Mee:ngs with Educa:on Department staff
• Two major themes: – In a :me of scarce resources, set research priori:es – Encourage greater collabora:on among researchers ies.ed.gov
3
Network Objec4ves 1. Focus resources and aZen:on on educa:on issues that are a high priority for the na:on 2. Create a structure and process for researchers to share ideas, build new knowledge, strengthen their research and dissemina:on capacity 3. Advance the field’s understanding of a problem or issue beyond what one research team can do on its own
ies.ed.gov
4
FY 2017 Network Topics 1. Scalable Strategies to Support College Comple4on – Develop and evaluate effects of programs or policies designed to increase the number of college students who earn degrees in open-‐ and broad-‐access ins:tu:ons
2. Exploring Science Teaching in Elementary School Classrooms Network – Uncover early elementary science teaching ac:ons associated with improved outcomes for students from low-‐income and underrepresented groups ies.ed.gov
5
Network Structure • A minimum of two Research Teams are required to form a Network. • Networks will comprise a maximum of four Science Teaching or five College Comple:on Research Teams • Network Lead will facilitate mee:ngs and coordinate Network ac:vi:es as well as conduct research ac:vi:es to complement Network studies ies.ed.gov
6
Network Structure Research Team*
Research Team*
Network Lead (1)
Research Team*
ies.ed.gov
7
*Although 4 research teams are depicted in this image, up to 4 Science Teaching teams and up to 5 College Comple:on teams may receive the grant. Each research team will carry out its own project.
Research Team*
Expecta:ons of Network Members • Share research plans, data collec:on tools, other informa:on and ideas • Provide construc:ve feedback • Look for opportuni:es to work together on: – Common measures – Research synthesis – Dissemina:on
ies.ed.gov
8
Requirements for the Network Lead 1. Administer and coordinate Network ac:vi:es 2. Conduct research to complement Research Team studies 3. Facilitate communica:on of Network findings
ies.ed.gov
9
Supplementary Ac:vi:es • IES will set aside up to $1 million for each Network to support collabora:on and foster new ideas • Funds are to support ac:vi:es carried out by two or more Network members. For example: – Develop common measures – Conduct exploratory research – Joint publica:on and dissemina:on ac:vi:es
• Supplementary ac:vi:es may be proposed star:ng year 2 and must be approved by beginning of year 5 ies.ed.gov
10
SECTION II: COLLEGE COMPLETION NETWORK • Guiding Objec:ves: – Focus on students at open-‐ and broad-‐access ins:tu:ons – Develop and evaluate strategies that ins:tu:ons can implement (beyond evalua:on seings) – Contribute to current college comple:on policy discourse from a research perspec:ve
ies.ed.gov
11
Teams: Substan:ve Concerns • Teams are free to propose and jus:fy their own substan:ve focus • The RFA lists 5 substan:ve focal areas: – Ins:tu:onal barriers; – Financial burdens; – Curriculum and pedagogy; – Insufficient advising; – Social/psychological factors.
• The Ins:tute encourages projects that focus on: – 1) ins:tu:onal barriers and/or free tui:on; – 2) comprehensive interven:on strategies
ies.ed.gov
12
Teams: General Requirements – Efficacy study: impacts and costs – Joint ac4vi4es: mee:ngs, research, dissemina:on – Sample: • Postsecondary students eligible for an AA or BA degree • Analysis of at least one subgroup of interest
– Outcomes: degree comple:on if possible; or progression :ed to degree comple:on – SeMng: open-‐ and broad-‐access ins:tu:ons ies.ed.gov
13
Team Narra:ve: Significance Sec:on – Requirement: This sec:on must describe the interven:on. – Recommenda4ons: • Primary barriers to degree comple:on • Collabora:on with systems and ins:tu:ons • Theory of ac:on for your strategy • Poten:al for scaling your strategy ies.ed.gov
14
Team Narra:ve: Research Plan • Requirement: the Research Plan sec:on must include these parts: – Sample and seing – Development process (if applicable) – Research design – Power analysis – Measures – Data analysis procedures – Cost analysis ies.ed.gov
15
Team Narra:ve: Research Plan – Recommenda4ons: • Development Process (if applicable) • Research Design: producing causal evidence • Design Elements: power analysis, outcome measures, mediators and moderators, fidelity of implementa:on, control group prac:ces, and data analysis procedures • Cost Analysis • Timeline ies.ed.gov
16
Team Narra:ve: Personnel – Requirement: Sec:on must list key personnel and their qualifica:ons – Recommenda4ons: • Qualifica4ons to carry out the proposed work • Sufficient 4me to carry out the proposed work • Management structure and organiza:on chart
ies.ed.gov
17
Team: Resources – Requirements: • Resources to conduct the project • Resources to disseminate the findings
– Recommenda4ons: • Grant management capacity (primary ins:tu:on) • Access to research resources from all sources (primary, subaward, outside) • Access to par4cipa4ng colleges and universi4es (leZers of agreement) • Access to data necessary to carry out the project ies.ed.gov
18
SECTION III: SCIENCE TEACHING NETWORK • Purpose: Uncover early elementary science teaching ac4ons associated with improved outcomes for students from low-‐income families and underrepresented groups • “Teaching ac:ons” – Something a teacher does during lessons that is observable and specific
• “Underrepresented” – Groups not enrolled in science courses or employed in science professions at levels expected given their numbers in the popula:on ies.ed.gov
19
Science Teaching Network: Ul4mate Objec4ve • To promote – more refined theore:cal constructs of teaching – beZer understanding of how it affects student outcomes – stronger measures of science teaching for use in research, prac:ce improvement, and high-‐stakes decision making – more ac:onable guidance, training, and supports for educators and teacher educators.
ies.ed.gov
20
Science Teaching Research Team: Roles and Responsibili:es • 3 complementary, prospec:ve studies: • Study 1: Describes specific teaching ac4ons, student responses to these ac:ons during science lessons, and the correla:ons of these with student outcomes • Study 2: Develops reliable and valid measures to capture these teaching ac:ons • Study 3: Follows students over mul4ple school years to explore the rela:ons between exposure to the specific observable teaching ac:ons and student outcomes.
ies.ed.gov
21
Science Teaching Research Team: General Requirements • Content Area – Science teaching, including physical, earth, and/or life sciences – Addi:onal content areas that may be related to science teaching, such as reading or math
• Sample – Early elementary school grades (i.e., first, second, and/or third grade) students and their science teachers – Low-‐income and/or from underrepresented groups ies.ed.gov
22
Science Teaching Research Team: General Requirements • Measures – Teaching theorized to influence student outcomes • e.g., observable science teacher ac:ons, classroom management and organiza:on, and teacher science content knowledge
– Students • demographic informa:on • knowledge of science concepts and prac:ces • student mo:va:on to learn science
ies.ed.gov
23
Science Teaching Research Team: General Requirements • SeMng – Authen4c educa4on seMngs – External providers, if working in partnerships with schools or school districts • E.g., museums, science centers, environmental educa:on centers, libraries, and industries
ies.ed.gov
24
Research Team Applica:on: Narra:ve Requirements • The 30-‐page project narra:ve must include 4 sec:ons: – Significance – Research Plan – Personnel – Resources
ies.ed.gov
25
Science Teaching Research Team: Significance • Provide ra4onale for your approach to addressing the objec4ves of the Network • Explain your vision for the 3 studies and the science teaching ac:ons you will address • Discuss the prac:cal importance of the proposed work and how it will lead to improvements in educa:on outcomes for students, especially low-‐ income and underrepresented students
ies.ed.gov
26
Science Teaching Research Team: Research Plan • Explain the methodology for each of the 3 studies • Each study must – Be prospec:ve – Include primary data collec:on – Focus on students from low-‐income families and underrepresented groups
ies.ed.gov
27
Science Teaching Research Team: Personnel • Iden4fy and describe the qualifica4ons of the members of the Research Team • For each Research Team member, describe her/his – Exper:se – Responsibili:es – Time commitments
ies.ed.gov
28
Science Teaching Research Team: Resources • Describe how you have the ins:tu:onal capacity to complete a project of this size and complexity • Describe the ins:tu:onal capacity to disseminate the results from each study
ies.ed.gov
29
Sec4on IV: Network Lead Roles & Responsibili:es • Network administra:on and coordina:on – Planning and facilita:ng Network mee:ngs – Coordina:ng Network supplementary ac:vi:es
• Research to complement network studies – Synthesis of findings from across the Network – Addi:onal Research Ac:vity
• Communica:on of Network findings – Developing and hos:ng a Network website – Organizing briefings and presenta:ons ies.ed.gov
30
Network Lead: General Requirements • 15-‐page Applica:on • The applica:on must include: – A plan for network administra:on and coordina:on; – A plan for research ac:vi:es to complement Research Team studies; – A plan for communica:on of network findings; – A descrip:on of personnel qualifica:ons; and – A descrip:on of organiza:onal resources. ies.ed.gov
31
Network Lead: Network Administra:on & Coordina:on • Describe your understanding of the network’s goals as they relate to the current fields of research, policy, and prac:ce • Describe your vision of how the Network Lead will help the network accomplish its goals • Discuss your plans for network mee:ngs – Including major goals for mee:ngs and how you will facilitate exchange of informa:on and ideas, review and feedback of drap measures and reports, and plan for new ac:vi:es and collabora:ons ies.ed.gov
32
Network Lead: Research to Complement Network Studies • Synthesis of Findings from across the Network – Explain your vision for a synthesis, including its main objec:ves and target audiences – Discuss how you will involve the Research Teams and external reviewers in developing this synthesis – Discuss how you would present and organize the informa:on in a way that will be useful to policymakers, prac::oners, and researchers – Provide a :meline for produc:on and comple:on of the synthesis ies.ed.gov
33
Network Lead: Research to Complement Network Studies • Addi:onal Research Ac:vity – Describe the ac:vity and the poten:al benefits to the network ac:vi:es and products as well as the field at large – Discuss your publica:on plans and target audiences – Discuss any plans to use alterna:ve formats to disseminate findings from the addi:onal research ac:vity – Provide a :meline for the comple:on
ies.ed.gov
34
Network Lead: Communica:on of Network Findings • Explain how you will work with the Research Teams to communicate broadly about the objec:ves, findings, and ac:vi:es of the Network • Discuss your ideas for using briefings and other forums • Discuss your vision for the network website • Discuss plans for any conferences the Network would plan and host, including purpose, size, and target audience.
ies.ed.gov
35
Network Lead: Personnel and Resources • Describe the qualifica:ons of key personnel • Describe the resources your university, research firm, or organiza:on will provide for the Network Lead role
ies.ed.gov
36
SECTION V: Applica:on Requirements • A PI/Project Team may only submit one Research Team applica:on. • Research Team applicants may apply for the Network Lead role.
ies.ed.gov
37
Important Dates & Deadlines Applica4on Le\er of Deadline Intent Due Date August 4, May 19, 2016 2016 4:30:00 PM DC Time
ies.ed.gov
Applica4on Possible Start Package Dates Posted May 19, January 1, 2017 2016 to July 1, 2017
38
Schedule of Network Mee:ngs Number of mee4ngs
ies.ed.gov
Loca4on
Year 1 2
At least one mee:ng in DC
Year 2 1
DC
Year 3 1
DC
Year 4 1
DC
Year 5 1
DC
39
Other Important Sec:ons of the Applica:on • • • • •
Appendix A Appendix B Appendix C Appendix D Budget & Budget Narra:ve
ies.ed.gov
40
Appendix A • Appendix A is reserved for resubmissions of applica5ons. • Applicants to the FY 2016 compe55on for the College Comple5on Network who are re-‐applying in FY 2017 should complete Appendix A. • Because the Science Teaching Network compe55on is new in FY2017, applicants to this compe55on should not complete Appendix A.
ies.ed.gov
41
Appendix B Page Limit: 15 • Figures, charts, or tables that supplement the project narra:ve • Timelines for the project (very useful) • Examples of measures to be used – E.g., tests, surveys, observa:on, and interview protocols
• Do NOT include narra5ve text ies.ed.gov
Appendix C Page Limit: 10 • Examples of materials used in the program or policy: – curriculum material – computer screen shots – training documents – assessment items – other materials
ies.ed.gov
43
Appendix D: LeZers of Agreement No Page Limit • LeZers of Agreement from all the research partners – Joint LeZer from key partners – Separate LeZers from other organiza:ons involved – LeZers should clearly state the organiza:on’s expected role in the partnership and their commitments to the project – Similar leZers from any consultants, districts, and schools taking part – LeZers from holders of data should make clear that the data described in the applica:on will be provided for the proposed use by the project ies.ed.gov
44
Appendix E: Data Management Plan (Required for Research Team Applica:ons)
Page Limit: 5 pages • Requirement: Research Team applica:ons under both networks must include a Data Management Plan in Appendix E. • Content: Include your data management plan in this sec:on. ies.ed.gov
45
Budget and Budget Narra:ve Network Role
Maximum Number of Awards
Maximum Grant Dura4on
Maximum Grant Awards
Research Team
4
5 years
$4,000,000
Network Lead
1
5 years
$2,000,000
ies.ed.gov
46
SECTION VI: PREPARING A RESPONSIVE, HIGH QUALITY APPLICATION • Download and complete the correct applica4on package from Grants.gov • Pay close aZen:on to the general-‐, topic-‐, and role-‐specific requirements • Do not exceed maximum award amounts • Comply with formaing and font-‐size requirements; page limits, allowable content; and required content for the applica:on • Submit your applica:ons no later than 4:30:00pm Washington, DC 4me on August 4, 2016 ies.ed.gov
47
Peer Review Standards & Review Office • Compliance screening for format requirements • Responsiveness screening for program requirements • Assignment to reviewers with substan:ve and methodological exper:se on your topic • Reviewers assign preliminary scores • Most compe::ve applica:ons go to full panel – Many panelists will be generalists to your topic – Panelists discuss each applica:on and assign final scores
• Funding decisions based on panel scores and available resources ies.ed.gov
48
Criteria and Scoring: Research Team • Proposals will be evaluated on 4 criteria: 1. 2. 3. 4.
Significance Research Plan Personnel Resources
• Pay close aZen:on to Recommenda4ons for a Strong Applica4on – Recommenda:ons derive from IES experience – Reviewers use the recommenda:ons to differen:ate high-‐ and low-‐quality applica:ons
ies.ed.gov
49
Criteria and Scoring: Network Lead • Proposals will be evaluated on 5 criteria:
– Plan for network administra:on and coordina:on – Plan for research ac:vi:es to complement Research Team studies – Plan for communica:on of network findings – Personnel qualifica:ons; and – Organiza:onal resources
• Pay close aZen:on to Recommenda4ons for a Strong Applica4on
– The recommenda:ons derive from IES experience – Reviewers use the recommenda:ons to differen:ate high-‐ and low-‐quality applica:ons
ies.ed.gov
50
Tips to Improve Your Score • Put yourself in the reviewer’s shoes
– Good, clear wri:ng makes a difference! – Use diagrams and figures to help explain complex ideas, such as conceptual frameworks – Do not use diagrams with microscopic print
• Make informa:on easy to find
– Follow the same headings and subheadings as the RFA – Use bullet points and boldface to underscore key points
• Seek others’ input before you submit your proposal
– IES program officer – Colleagues who did not work with you on the proposal – Proofreader
ies.ed.gov
51
Informa:on Sources • Request for Applica:ons – hZp://ies.ed.gov/funding/
• Abstracts of Projects – hZp://ies.ed.gov/funding/grantsearch/index.asp
• Applica:on Package – www.grants.gov
• Program Officers –
[email protected] – Wai-‐
[email protected] ies.ed.gov
52