In 2014 Makel en Plucker found out that less than 1% of the studies published in the top journals for education research were replication studies. Did things get better afterwards? This is what Perry, Morris & Lea tried to find out. And the answer is yes, but very, very, very slightly.
Publication Year | Total publications | Replication studies | Direct Replications | ||
---|---|---|---|---|---|
Freq. | Freq. | % | Freq. | % | |
2011 | 18,534 | 24 | 0.13 | 8 | 0.04 |
2012 | 19,814 | 27 | 0.14 | 8 | 0.04 |
2013 | 20,110 | 29 | 0.14 | 8 | 0.04 |
2014 | 20,009 | 39 | 0.19 | 13 | 0.06 |
2015 | 20,189 | 43 | 0.21 | 7 | 0.03 |
2016 | 21,870 | 43 | 0.20 | 13 | 0.06 |
2017 | 22,285 | 43 | 0.19 | 17 | 0.08 |
2018 | 23,542 | 52 | 0.22 | 23 | 0.10 |
2019 | 29,355 | 61 | 0.21 | 21 | 0.07 |
2020 | 30,336 | 81 | 0.27 | 27 | 0.09 |
Total | 226,044 | 442 | 0.20 | 145 | 0.06 |
No really:
This just ain’t good. Psychology as a science has gone through a large self-correcting replication crisis, making psychology as science better.
Abstract of “A decade of replication study in education? A mapping review (2011–2020)”:
Replication studies in education are relatively rare. Of the few which are conducted, many are conceptual rather than direct replications. With so few replication studies, and many of those that are attempted producing null results, the scientific status of the evidence base for educational policy and practice is in question. Replicating Makel and Plucker’s review of the education replication literature, conducted in 2014, this paper presents a mapping review looking at rates of replication in education research from 2011 to 2020. We provide an overview of the number of replication studies by replication type, year, outcome, authorship, and journal. Our results are consistent with those of Makel and Plucker, revealing very low but gradually increasing rates of replication study in education. We discuss the role of replication in producing a robust and trustworthy evidence base for policy and practice, and some of the challenges in operationalising definitions of replication we encountered.
Reblogged this on kadir kozan.