This post is really about two things. On the one hand, the new meta-analysis examines the effect of physical activity on cognition. But there is something else I need to address first. One of the most-read pieces on my blog at the moment asks a simple question: how reliable are most meta-analyses in educational research? In essence, that post is a tour of all the ways things can go wrong. And often do. I received many responses, but one recurring undertone was: “meta-analyses are bad research”. That was certainly not what the authors of the review I was commenting on intended. Quite the opposite, I think. I read their work mainly as a plea to do things better. And this new meta-analysis on movement and cognition shows that it is indeed possible.
The study by Bartoš and colleagues (not yet peer-reviewed, but impressively rigorous) is a meta-meta-analysis of the effects of physical exercise on cognition, memory and executive functions. In recent years, numerous optimistic reviews have appeared, often making bold claims along the lines of “exercise makes you smarter”. But if you really want to know what happens when you break down hundreds of meta-analyses to the level of individual studies, this preprint offers a fascinating answer. There is a great deal of noise, even more heterogeneity, and quite a lot of publication bias. And that is precisely what makes this preprint such a model of how things should be done.
The authors re-analysed 215 meta-analyses and reconstructed 2,239 effect sizes at study level. They did not rely on a single model, but used several robust approaches side by side: Bayesian model averaging, selection models, PET-PEESE and weight functions. They did not stop at funnel plots; they also used z-curves and bias factors to provide convincing evidence. And instead of settling for a single average effect, they report prediction intervals that reveal the true breadth of the underlying results. The outcome is both reassuring and sobering. Some forms of physical activity show small positive effects in specific populations. Still, the broader claim that “exercise improves cognition” hardly holds up once publication bias and heterogeneity are taken seriously.
What the authors do particularly well is avoid confusing a tidy average with the actual truth. The mean effect for general cognition shrinks to about a quarter of a standard deviation after corrections. For memory and executive functions, it is reduced to almost nothing. More importantly, the prediction intervals range from potential harm to potential benefit. Once you see that spread, it becomes clear why such averages should not be overinterpreted. This is precisely what methodological textbooks – and the earlier review I wrote about – try to explain. But we still too often overlook it in practice.
There is something else I appreciate: the work’s openness. The authors publish their code and data. They explicitly acknowledge limitations (such as possible overlap between studies). And they even correct previously published meta-analyses that had reported inflated effects. That is how it should be. And yes, it is still a preprint, so that things may shift during peer review. But as an example of transparent, rigorous and methodologically consistent meta-analysis, it already deserves a place in the category of “model cases”.
Finally, a word about the overall conclusion of this enormous piece of work. It is somewhat less exciting than some policy documents might prefer. Exercise is healthy for countless reasons, but the claim that it gives the brain a general cognitive boost is far less solid than often assumed. It may not be the sort of message that goes viral, but it is one that helps us understand research a little better. And that is valuable in itself.
And because I know the question will come otherwise: what about active or movement-based learning? This meta-meta-analysis does not address that directly. But it does nuance one of the assumptions behind it: that movement is a cognitive turbocharger. The authors show that the cognitive effect of physical activity is small, uncertain and extremely heterogeneous.
Image: https://www.pexels.com/nl-nl/foto/ontspanning-kamer-vreugde-plezier-8613312/