Replication in educational science? Things are getting very slightly better (but it’s still very bad))

In 2014 Makel en Plucker found out that less than 1% of the studies published in the top journals for education research were replication studies. Did things get better afterwards? This is what Perry, Morris & Lea tried to find out. And the answer is yes, but very, very, very slightly.

Publication Year Total publications Replication studies Direct Replications
Freq. Freq. % Freq. %
2011 18,534 24 0.13 8 0.04
2012 19,814 27 0.14 8 0.04
2013 20,110 29 0.14 8 0.04
2014 20,009 39 0.19 13 0.06
2015 20,189 43 0.21 7 0.03
2016 21,870 43 0.20 13 0.06
2017 22,285 43 0.19 17 0.08
2018 23,542 52 0.22 23 0.10
2019 29,355 61 0.21 21 0.07
2020 30,336 81 0.27 27 0.09
Total 226,044 442 0.20 145 0.06

No really:

This just ain’t good. Psychology as a science has gone through a large self-correcting replication crisis, making psychology as science better.

Abstract of “A decade of replication study in education? A mapping review (2011–2020)”:

Replication studies in education are relatively rare. Of the few which are conducted, many are conceptual rather than direct replications. With so few replication studies, and many of those that are attempted producing null results, the scientific status of the evidence base for educational policy and practice is in question. Replicating Makel and Plucker’s review of the education replication literature, conducted in 2014, this paper presents a mapping review looking at rates of replication in education research from 2011 to 2020. We provide an overview of the number of replication studies by replication type, year, outcome, authorship, and journal. Our results are consistent with those of Makel and Plucker, revealing very low but gradually increasing rates of replication study in education. We discuss the role of replication in producing a robust and trustworthy evidence base for policy and practice, and some of the challenges in operationalising definitions of replication we encountered.

One thought on “Replication in educational science? Things are getting very slightly better (but it’s still very bad))

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.