This paper by Lin Zhang, Paul A. Kirschner, William W. Cobern & John Sweller makes a clear but bold statement already in the title: “There is an Evidence Crisis in Science Educational Policy“. The abstract of the open-access paper elaborates:
There is a considerable gap between many of the findings from educational psychology research and educational practice. This gap is especially notable in the field of science education. In this article, the implications of three categories of research and their findings for science educational policy in the USA and other jurisdictions were reviewed. We indicate that a particular category of research that we call “Program-Based Studies,” has dominated the formulation of educational standards while a large number of critical findings from randomized, controlled studies and correlational studies that overwhelmingly show minimal support for the suggested policy have been marked as irrelevant and excluded. The current blanket-emphasis on program-based studies at the expense of the other types of research is misplaced. Educational standards should represent a balanced view of the available data including findings from controlled and correlational studies. Finally, we indicate how these different forms of research might inform each other and provide coherent and consistent implications for educational procedures.
The paper focuses more specifically on science education and the gap between what research shows and policy, but also and mentions a very alarming finding near the conclusion I do want to highlight:
In a recent review, Hedges and Schauer (2018) noted that there has been a dramatic decline in the use of randomized, controlled trials in education in the USA between 1980 and 2000. Specifically, in the field of science education, Taylor et al. (2016) found that a vast majority of impact studies were not conducive to quantitative synthesis. A review of research about inquiry-based science instruction by Minner et al. (2010) found that about 78% of the included studies either did not have a comparison group or used non-equivalent control groups. Our review is consistent with these reviews, finding evidence that controlled studies are underrepresented in educational standards and reports.
This is a replication crisis waiting to happen, IMHO…