As I wrote yesterday, last week I spent time with several other researchers (and policymakers too, by the way) thinking about the worldwide decline in performance on PISA, PIRLS, TIMSS… Strikingly little attention was paid to teaching and instruction, by the way. I deliberately did not write “learning outcomes”. This is because that touches on one of the recurring themes running through the different discussions.

Alongside all the possible explanations that came up, someone would repeatedly raise the uncomfortable question: are tests such as PISA and the other international comparisons still fit for our times?

Nobody wanted to throw international comparisons in the bin. But one person put the question like this: PISA was developed more than 25 years ago. So why should it still fit today’s reality? In doing so, they rather strikingly ignored the fact that PISA adds new elements with every round. Also, PISA does actually evolve. Someone else went even further and wondered aloud whether we might now be living in a post-writing era. Perhaps texts simply aren’t that important anymore?

In response, researchers pointed, for example, to the strong correlation between reading and listening comprehension. At the same time, the importance of prior and background knowledge was widely acknowledged. But the question stayed with me.

I immediately thought of a thermometer. It can tell you that you have an elevated temperature, or perhaps even a fever. The thing doesn’t tell you whether you have the flu or not. It reveals a symptom, but, just as with PISA and the others, the possible cause is difficult to establish. Should we replace the thermometer if it turns out that we are constantly walking around with a fever?

Mind you, I have noticed that there are several modern types of thermometers. I have been using an electronic one myself for quite some time. However, during COVID the highly specialised versions suddenly appeared. Just as PISA, PIRLS, and TIMSS evolve, so do we.

But like every metaphor, this comparison also has its limitations. We pretty much agree on body temperature. In fact, PISA and the others are more like a collection of thermometers. The difficult question was mainly whether that temperature is still so important. Perhaps today we should be looking at something else instead.

But precisely because of that last remark, I think I know my personal answer.

Perhaps basic skills are really comparable to our body temperature. If something goes wrong with them, all kinds of things can go wrong in our society. Without reading comprehension, acquiring information and, by extension, knowledge will not go well. This is true whether on paper, on a screen or in a video.

After a previous bad round of PIRLS or PISA, a Dutch professor, who has since retired, asked me whether it really is such a bad thing if poor children can no longer read. I thought this person was joking, but it turned out they weren’t.

I don’t think I am anywhere near ready to leave behind the ambition of democratisation. And that means you need tools to map the situation.

Mind you, every instrument has its limitations, and you need to know them well.

But don’t just shoot the messenger…