I want to discuss a new study on PIAAC, the lesser-known adult sibling of PISA. PIAAC measures, among other things, literacy and numeracy among adults, while PISA assesses related skills among 15-year-olds.

There was something in the Flemish results in 2024 that fascinated me. I wrote on my Dutch blog:

By 2012, average PISA scores in Flanders were already clearly declining, yet that decline is not visible in the 2023 PIAAC scores of 25- to 34-year-olds.

You have to admit, that seems strange. If PISA tells us something about the skills young people acquire at school and will later need in adult life, you might expect a decline there to become visible somewhere down the line when we measure adults’ skills.

At the time, I suggested several possible explanations. People may continue to develop their skills after the age of 15 through further education, training and work. And, of course, there is another explanation that may become particularly relevant when the next PISA results arrive: perhaps it also matters how seriously young people take a test such as PISA.

A day later, I added an important warning about PIAAC itself in a Dutch follow-up post. Response rates were quite low in many countries. Flanders and the Netherlands, for example, received a medium caution from the OECD. Researchers use statistical methods to correct for possible non-response bias, but they cannot guarantee that all bias has disappeared.

So why am I returning to two blog posts from almost two years ago on my Dutch-language blog? Not because another round of PISA results is approaching withn two weeks, but because the OECD has now published a new working paper with the very appropriate title Towards a stronger link between PISA and PIAAC: Two assessments, one story?

And the paper sheds new light on precisely the puzzle that caught my attention then. The short answer to the question in the title seems to be: sometimes, but certainly not always. The researchers compare PISA and PIAAC in several ways. You can simply ask whether countries that perform well in PISA also perform well in PIAAC. There is indeed a relationship here, and please note that the crucial word is relationship. The association becomes stronger when the PIAAC sample is restricted to 16- to 19-year-olds, excluding recent immigrants. The correlation between average scores is .74 for mathematics/numeracy and .53 for reading/literacy. Especially the first is a strong correlation. Countries where young people perform well in PISA therefore tend to have relatively high-performing young adults in PIAAC too.

But that is not our puzzle. We can ask a different question. Not only do the same countries score highly?, but if a country’s performance rises or falls over time in PISA, do we see the same movement in PIAAC?

And suddenly the picture changes quite dramatically. When we move from the photograph to the film, the relationship between trends in the two assessments turns out to be generally very weak.

This remains the case when the researchers restrict the PIAAC sample to 16- to 19-year-olds and exclude recent immigrants. While PISA shows a broadly downward movement across many countries, changes in PIAAC are much more mixed. And some countries make this discrepancy particularly visible.

Finland is perhaps the most striking example. In one comparison, the country shows a clearly negative PISA trend but a strongly positive PIAAC trend. Poland and Lithuania are outliers in a different direction. Remove these three striking cases from the analysis and the relationship between the two trends becomes somewhat more positive, but it remains modest and statistically quite uncertain.

Norway also stands out. Finland, Norway and the Slovak Republic perform better in PIAAC than we would expect on the basis of their PISA performance. For the Netherlands, the same is true specifically for literacy. On the other side, we find countries including New Zealand, Poland and the United States, which perform worse in PIAAC than their PISA results would lead us to expect.

So do people in some countries somehow become ‘less smart’ after leaving school, while the opposite happens elsewhere?

Obviously, it is not that simple. Still, all of this makes what I wrote about Flanders in 2024 a little less puzzling. The Flemish pattern is apparently not unique. Flanders is not reported separately in this particular OECD paper, so the new analysis does not allow me to add much directly to my original observation. But the broader phenomenon is clearly not confined to Flanders.

Which leaves the question: why might PISA and PIAAC diverge? One possible explanation is actually quite straightforward: people do not stop learning when they turn 15, and what happens after that age can differ between countries. What young people acquire during compulsory education certainly helps shape their later skills, but it does not determine them completely. Skills can continue to develop through further education, t,raining and work, or they can decline when they are not used.

But there are other possible explanations. PISA and PIAAC do not measure exactly the same thing, they assess different populations, and they do so under very different circumstances. PISA is administered to 15-year-olds at school, while PIAAC is administered to adults in their homes. Test duration, sampling and participant motivation also differ. Even mathematics and numeracy are not identical constructs: PISA, for example, includes more formal, school-based mathematical content than PIAAC.

And yes, this is where the warning from my second 2024 blog post reappears. The authors explicitly discuss differences in response rates between countries. In PIAAC Cycle 2, these ranged from 27 to 73 per cent. The OECD uses extensive procedures to investigate and correct for possible bias, but the authors also point out that such adjustments cannot necessarily eliminate all differences between countries and surveys.

The researchers therefore warn us not to overinterpret comparisons between trends. Trend estimates are less reliable than comparisons of average performance levels between countries, and differences in participation, test administration or motivation could affect countries differently.

So, I’ll try an attempt at a conclusion, without turning this into another round of OECD bashing. If you take a photograph, PISA and PIAAC quite often tell the same story. Countries with strong performance among 15-year-olds generally also have relatively strong performance among young adults. So far, so good.

But if you watch the film, things become much more complicated. A country that declines in PISA does not necessarily decline in the same direction or at the same pace in PIAAC. It may even move in the opposite direction. Finland and Norway make this particularly visible, and I think the Flemish puzzle I described in 2024 fits into this broader picture as well.

None of this means that PISA tells us nothing about skills that matter later in life. Quite the opposite: the association between performance levels is far too clear for that conclusion. But neither can we simply look at a rising or falling PISA score and infer what will happen to the skills of adults several years later.

And that may be a crucial nuance to keep in mind when the next PISA results arrive.