When I discussed the (very) critical study of Hattie’s work last week, I removed a passage at the very last minute. Not because I wanted to censor anything. But because I wasn’t sure yet whether I agreed with that particular criticism or not. More specifically, Kalmendal Bruns and colleagues pointed out that much of the research Hattie used in his analyses, for example on technology in education, might be outdated. So was it really a good idea to keep using those older meta-analyses? Now, the authors themselves do make a distinction here. They don’t argue that all older research is useless, but that some topics, such as educational technology, need updating more frequently.

At first sight, this seems logical. But is it? I’m not so sure. We like to think that the world and people change. And partly, of course, they do. But a lot also stays the same. Do we learn differently today than we did in the past? In many ways, I don’t think so. But of course, we now have technology that can do things people could only dream of 50 years ago.

But at the same time… do the underlying principles really change that quickly? If you look at Audrey Watters’ rather fantastic work on Teaching Machines, her history of educational technology starts much earlier than you might think (at the beginning of the previous century) and ends much earlier too (in the 1980s). The latter is because many of the principles we still recognise today were already there. Even when you look at things like Duolingo, AI tutors, etc., you see many similarities. That doesn’t mean, of course, that the effects of these different technologies are necessarily the same, or that they will stay the same. But perhaps today’s technology has simply become better at doing these things? Perhaps. Let’s hope so. Although we also know something called enshittification, and some research suggests some steps forward are actually steps back!

All this doesn’t mean, of course, that we should ignore newer meta-analyses when they are available. But even then, I think the question is mainly what these newer studies add or change. Personally, I think we should look at each study and ask whether it is still relevant. Not just whether it is ‘old’, but also whether it was conducted properly (after all, our standards can change).

I remember a fellow researcher once telling me you shouldn’t cite research more than 10 years old. Just imagine how much we would lose if we followed that rule! But only referring to old research is also wrong, in my opinion. So I have no problem referring to Ebbinghaus (1885) or Thorndike, although I will usually also mention that more recent research still supports their findings. Just because something is old doesn’t mean it is wrong. But being old doesn’t make it right either. Think, for example, of certain ideas of Piaget that have been revised or even refuted by later research.