In 2014 Makel en Plucker found out that less than 1% of the studies published in the top journals for education research were replication studies. Did things get better afterwards? This is what Perry, Morris & Lea tried to find out. And the answer is yes, but very, very, very slightly.
| Publication Year | Total publications | Replication studies | Direct Replications | ||
|---|---|---|---|---|---|
| Freq. | Freq. | % | Freq. | % | |
| 2011 | 18,534 | 24 | 0.13 | 8 | 0.04 |
| 2012 | 19,814 | 27 | 0.14 | 8 | 0.04 |
| 2013 | 20,110 | 29 | 0.14 | 8 | 0.04 |
| 2014 | 20,009 | 39 | 0.19 | 13 | 0.06 |
| 2015 | 20,189 | 43 | 0.21 | 7 | 0.03 |
| 2016 | 21,870 | 43 | 0.20 | 13 | 0.06 |
| 2017 | 22,285 | 43 | 0.19 | 17 | 0.08 |
| 2018 | 23,542 | 52 | 0.22 | 23 | 0.10 |
| 2019 | 29,355 | 61 | 0.21 | 21 | 0.07 |
| 2020 | 30,336 | 81 | 0.27 | 27 | 0.09 |
| Total | 226,044 | 442 | 0.20 | 145 | 0.06 |
No really:
This just ain’t good. Psychology as a science has gone through a large self-correcting replication crisis, making psychology as science better.
Abstract of “A decade of replication study in education? A mapping review (2011–2020)”:
Replication studies in education are relatively rare. Of the few which are conducted, many are conceptual rather than direct replications. With so few replication studies, and many of those that are attempted producing null results, the scientific status of the evidence base for educational policy and practice is in question. Replicating Makel and Plucker’s review of the education replication literature, conducted in 2014, this paper presents a mapping review looking at rates of replication in education research from 2011 to 2020. We provide an overview of the number of replication studies by replication type, year, outcome, authorship, and journal. Our results are consistent with those of Makel and Plucker, revealing very low but gradually increasing rates of replication study in education. We discuss the role of replication in producing a robust and trustworthy evidence base for policy and practice, and some of the challenges in operationalising definitions of replication we encountered.
Reblogged this on kadir kozan.
[…] Jelte Wicherts en Janneke van de Pol, besloot dit uit te zoeken. Niet met een “nieuw model”, maar met iets wat in de onderwijswetenschap nog te weinig gebeurt: een directe replicatie. Ze reconstrueerden het klassieke experiment van David Wood en collega’s uit 1978, dat vaak wordt […]
[…] Jelte Wicherts and Janneke van de Pol, decided to find out. Not by proposing a “new model”, but by doing something that still happens far too rarely in education research: a direct replication. They reconstructed the classic 1978 experiment by David Wood and colleagues, often cited as the […]