Guidelines for Effective Education Research and Development

Common Guidelines for Education Research and Development Common Guidelines for Education Research and Development Edith Gummer MSP Conference 12/16/2013

Knowledge development is not linear, but multi- directional Scientific contributions by multiple researchers, research teams and practitioners are needed An efficient research and development program may require fewer studies as one moves from foundational to impact evaluation studies Implementation research is important for design and development projects and for impact evaluations Outcomes may be education and/or learning in multiple settings 2

Improve the quality and pace of findings from education research and development proposals Develop an education infrastructure that supports more rapid and efficient knowledge development Aid NSF and ED in making informed choices about where to invest scarce research and development dollars Provide clarity for the field (and within the two agencies) 3

1. Proposals that seek to test, develop, or refine theories of education and learning that advance knowledge acquisition and the factors that facilitate or impede learning 2. Proposals that design and test new approaches to teaching and learning 3. Proposals that investigate some application of practice (e.g.,a new curriculum, a new technology) 4. Proposals that study the impact of an education practice or policy (e.g., what effects are College and Career ready standards in mathematics having on instruction?) 4

Foundational research and development Fundamental knowledge that may contribute to teaching and/or learning Early stage/exploratory Examines relationships among constructs to establish logical connections Design and development Design and iteratively develop particular interventions (programs, policies, practices or technologies); can also pilot test fully developed intervention to see if it achieves its intended outcomes 5

IMPACT STUDIES Efficacy Studies Estimate the impacts of strategies under optimal conditions of implementation Effectiveness Studies Examine implementation and estimate impacts similar to routine practice but still on a limited scale Scale-up Studies Explore implementation and estimates impacts under conditions that prevail under wide-scale adoption 6

Introductory material NSF and ED missions- distinct but complementary Guidelines for six types of research and development proposals Foundational Early Stage and Exploratory Design and Development Impact Studies Efficacy Effectiveness Scale-up Conclusions and recommendations for use 7

Section Purpose Section Heading Heading Content & Example Gives an overview of the Study Type. No subheadings Content & Example Subheadings Subheadings Justification- Entry Significance for policy and/or practice Basis in prior theory & evidence Evidence- Exit Likely project outcomes Plans for the research & development Plan for external feedback 8

Ex Officio: Joan Ferrini-Mundy Assistant Director, EHR and John Easton, Director, Institute of Education Sciences Co-Chairs: Janice Earle, EHR and Rebecca Maynard, National Center for Evaluation and Regional Assistance, ED Members: ED: Elizabeth Albro, IES; Karen Cator, Office of Technology; Michael Lach, Office of the Secretary; Loy Lesnick, Center for Evaluation and Regional Assistance; Ruth Curran Neild, National Center for Evaluation and Regional Assistance; Lynn Okagaki, IES; Jefferson Pestronk, Office of Innovation and Improvement; Anne Ricciuti, IES; Allen Ruby, IES; Deborah Speece, National Center for Special Education. NSF: Jinfa Cai, EHR-DRL; Edith Gummer, EHR-DRL; Jim Hamos, EHR-DUE; Gavin Fulmer, EHR-DRL;Janet Kolodner CISE and EHR-DRL; Susan Winter- SBE 9

These Guidelines are most relevant for NSF programs and projects that undertake education research and development activities. Some solicitations will explicitly reference the Guidelines. Not intended for outreach or scholarship activities. 10

The Guidelines are not intended to supplant, but are criteria. One element of the intellectual merit criterion for proposals is whether the project can advance knowledge and understanding. In addition, the intellectual merit criterion calls for a well-reasoned, well organized plan based on a sound rationale along with a mechanism to assure success. are consistent with consistent with the Merit Review 11

No.The Guidelines do not preclude or favor any research methods, but they do underscore the importance of ensuring that the methods are well described, justified, and appropriate to the research questions that are posed. Qualitative and quantitative approaches may be used in all of the six research genres that are described in the Guidelines. 12

Reviewers will be informed of the Guidelines through multiple approaches. The Guidelines will be: posted on the NSF website, referred to in program solicitations, discussed in reviewer webinars and orientations, and presented at PI and other professional meetings. 13

The Guidelines include recommendations for all types of studies that call for external feedback on the work being proposed. External feedback can include a number of approaches including third party evaluation, program officer evaluation, and/or regular feedback from advisory groups. It is up to the proposer to explain (a) why the external feedback planned is appropriate, and (b) how it is aligned with program requirements. 14

No. The Guidelines are intended to help PIs in proposal preparation. The key point of the Guidelines is to ensure that projects are explicit about their research questions, methods and analytic approaches in their proposals. These criteria should be relevant for all types of education R&D efforts. 15

FAQs on the Common Guidelines available on NSF website: NSF-127: http://www.nsf.gov/pubs/2013/nsf13127/nsf 13127.pdf Point of contact for NSF: Janice Earle jearle@nsf.gov 16