One of the most striking conclusions from PISA 2025 may be wrong. A new analysis suggests that the apparent sharp decline among more advantaged students may largely be the result of a change in measurement.

It was one of the most important new insights from PISA earlier this month: it is now mainly children from wealthier, more highly educated families who are falling behind worldwide. I even mentioned this finding in a meeting earlier this week. But… is it actually true? Marc Colomer and colleagues have just published a working paper that seriously calls the whole thing into question.

Within PISA, the OECD uses the well-known ESCS index to measure the socioeconomic status (SES) of the participating 15-year-olds. ESCS stands for the Index of Economic, Social and Cultural Status. It consists of three components: parents’ occupations, their educational attainment, and possessions in the home. But that last component, HOMEPOS, was measured quite differently in 2025 than in 2022. The questions about parents’ education also changed.

In 2022, for example, PISA asked about different types of books in the home: classic literature, contemporary literature, scientific books, art and music books, technical reference books, dictionaries, and books that help with schoolwork. These questions appear to have disappeared in 2025.

They were replaced by new, nationally selected items. Think of things such as running water, a parking space at home, rubbish collection, a refrigerator, air conditioning or heating, a swimming pool, a security system, a smart TV, a dishwasher, or domestic help.

Mind you, there may be good reasons for doing this. The authors themselves stress that the new questions may actually be better if you want to capture young people’s material living conditions. But… this also makes it much harder to compare the 2022 and 2025 scores. Because it turns out that the old and new items relate very differently to academic performance.

The eleven items that disappeared after 2022 had an average correlation of .201 with mathematics scores. For the sixteen items included in both assessments, the correlations were .122 in 2022 and .108 in 2025. For the new items introduced in 2025, it was only .052. Even after taking parents’ education and occupation into account, the difference remains substantial.

When you look at the changes, this does not seem particularly surprising. In 2022, the measure contained relatively more information about cultural capital, which was strongly related to academic performance. In 2025, it contained more information about material living conditions which, particularly in many wealthy OECD countries, are much less useful for distinguishing between students in terms of academic performance. But this also changed which students ended up in the highest socioeconomic quartile.

The researchers show that this actually happened. When they classify the same students using the official measure and their harmonised measure, 34 per cent of students change HOMEPOS quartile in 2022. The researchers argue that this creates the impression that the most advantaged students have suddenly fallen dramatically behind.

The authors therefore try to use the same measures in their analyses wherever possible. They reconstruct a socioeconomic index using the sixteen possession items present in both 2022 and 2025, calibrate them jointly, and scale the other components to a common scale as much as possible. When combined, the three components continue to follow the OECD methodology. This newly developed ESCS index by the team also correlates at .971 with the official ESCS. So it is not as if they have invented a completely different definition of socioeconomic status. This is strong work, imho.

Using the official ESCS, the four socioeconomic quartiles declined in mathematics by 2.8, 3.9, 2.6 and 14 points respectively. The gap between the top and bottom quartiles therefore narrowed by 11.2 points.

Using the harmonised index, the same four groups decline by 7.7, 5.2, 4.0 and 6.2 points. The gap then does not narrow at all. It widens by 1.6 points.

When the researchers use ten groups instead of quartiles, the issue becomes even more visible. With the harmonised index, the striking kink at the top almost completely disappears. All groups decline within a much narrower range. You might still wonder whether a few unusual countries are driving the result. But that does not seem to be the case either.

When they use the official ESCS, the mathematics gap narrows in 31 of the 36 education systems studied, with the change statistically significant in 19. But when the team uses the harmonised index, the gap still narrows in 17 systems, but in only one of them is the change statistically significant. In two countries, the gap even widens significantly! Moreover, in all 36 education systems, the official highest-SES group shows a larger decline than it does when they use the harmonised measure.

But the researchers wanted to check further. They therefore went back to PISA 2015 and calculated indices for 2015, 2018, 2022, and 2025 using the same set of items. According to the published indices, the socioeconomic gap appears to increase initially and then shrink sharply in 2025. When the same measurement rule is used across all four assessments, that spectacular narrowing disappears. According to this harmonised ESCS, the socioeconomic gap in mathematics is actually 6.5 points larger in 2025 than it was in 2015.

The researchers also found a second measurement problem. The OECD also changed the questions about parents’ education in 2025. In 2022, PISA asked students separately about their mother and father. In 2025, the questions referred to parent or guardian 1 and parent or guardian 2. Again, a change that fits the times. The response categories also changed, and there are many more missing responses. Moreover, the original questions are not included in the public dataset, making it harder for the researchers to reconstruct this particular problem.

But when they simply look at students whose parents completed tertiary (higher) education, their average mathematics performance barely changes between 2022 and 2025: +2 points. Students whose parents completed no more than twelve years of education, by contrast, decline by 8.2 points.

Finally, there is an almost ironic detail in this whole story. The PISA 2025 Technical Report itself warns that, unlike in previous cycles, ESCS in 2025 should not be treated as a trend scale. And this is precisely because of the substantial changes in how PISA measures parental education, occupation, and home possessions. This does make you wonder why Schleicher and co. subsequently drew fairly strong conclusions from comparisons between socioeconomic groups in 2022 and 2025.

This does not mean that PISA 2025 is now worthless. These researchers do not challenge the overall decline in performance with their analyses either. And this working paper is probably not the final word. It dates from 22 September 2026 and has not yet been peer reviewed. The researchers also had to make their own choices to make the different PISA cycles comparable. They explicitly state that their harmonised index is not necessarily the one and only ‘correct’ way to measure socioeconomic status.

At the same time, using the OECD’s own data, they point to a pretty important problem. And that means that one of the most important new insights from PISA 2025 suddenly looks a lot less certain.