Sometimes it is frustrating that PISA data can only show us correlations. Not because correlations are worthless. Although the posts in which I keep hammering on about the difference may sometimes make it seem that way, some correlations almost beg for an explanation. A new study by Yifan Wang, Yanru Liu and Dennis Alonzo, published in the British Journal of Educational Psychology, is a wonderful and frustrating example. The researchers combined four rounds of PISA, data from more than 1.37 million 15-year-olds in 73 countries, and measures of economic inequality. Moreover, they found something you would not immediately expect: the greater the economic inequality in a country, the more academic perseverance young people reported. And if that did not raise enough questions already, this association was also stronger among young people from lower socioeconomic-status families.
By academic perseverance, the researchers mean students’ tendency to finish schoolwork and keep going when faced with distractions, difficulties, or challenges. The concept is closely related to grit. However, the researchers focus specifically on perseverance of effort. And there is something to be said for that: previous research suggests that this particular component is more strongly associated with academic performance. It is more strongly associated than *grit* as a whole.
Four times the same strange association
The researchers used the PISA 2000, 2012, 2018, and 2022 datasets. These years were not chosen at random as these four PISA rounds contained usable questions on perseverance. Altogether, the analysis includes 1,374,393 students from 73 countries, although the precise number of countries varies from one PISA round to the next.
The researchers then linked these data to the Gini coefficient. This is a commonly used measure of economic inequality. Each time, they used a previously measured Gini score. For PISA 2022, for example, they used inequality from 2020; for PISA 2018, they used the figure from 2016. A higher Gini score, incidentally, means greater economic inequality.
And then you get correlations that simply beg for an explanation. At the country level, the association between economic inequality and average academic perseverance was:
- PISA 2022: r = .61
- PISA 2018: r = .47
- PISA 2012: r = .48
- PISA 2000: r = .49
Four times positive, four times relatively strong. So countries with greater economic inequality had, on average, students who themselves reported persevering more. In addition, even when the researchers took into account GDP per capita, population size, and unemployment in a country or region, this association remained.
But wait a minute
There is, of course, a huge warning needed here. And I’m not just talking about the correlation bit. One needs to remember that we’re talking about countries that differ from each other in countless ways. The fact that two characteristics are associated at country level certainly does not mean that one causes the other.
The researchers obviously know this too. That is why their analyses go beyond these simple correlations. They use multilevel models in which students are nested within schools and schools within countries and control for various characteristics of both countries and students. They also take the PISA sampling design into account.
That makes the association, not unexpectedly, much smaller. Economic inequality on its own explains about 0.8 to 1.9 per cent of the differences in perseverance. So that is a very different story from the striking correlations between country averages. But more importantly, the positive association does not disappear. In all four PISA rounds, the researchers found that greater economic inequality is associated with higher reported academic perseverance.
And Wang and colleagues really go the extra mile to check whether they can break the result. They test different statistical models and time intervals for the Gini coefficient, and control for cultural differences and for differences between countries in people’s tendency to give extreme or acquiescent responses to questionnaires. They run analyses leaving out one country at a time and analyses excluding Latin American and Caribbean countries. But… the general pattern turns out to be pretty robust.
And then it gets even stranger
Because the researchers also looked at the students’ socioeconomic backgrounds. In three of the four PISA rounds (2012, 2018, and 2022, but not 2000), they found an interaction with SES. Notably, the positive association between economic inequality and perseverance was about 1.5 to 1.8 times stronger among students with lower SES. It was stronger than among students with higher SES.
This finding is both interesting and problematic. It would be easy to conclude that economic inequality helps disadvantaged youth persist. But we definitely do not have to check that hypothesis in any RCT soon. This paper proves nothing like that either.
The measures of perseverance are also based on self-reports from the 15-year-olds who participate in PISA. As said, we do not know for sure that economic inequality is the cause. And even if future research could demonstrate a causal relationship, we would still not know whether the underlying mechanism is something we should be enthusiastic about.
So why do they find these results?
The authors discuss several possibilities to explain the correlation they found. People may more often see education as a way to improve their lives or as a possible way to avoid falling behind when they live in a country or region with more inequality. Such greater inequality may also trigger more competition and/or status comparison. Therefore, parents may place a stronger emphasis on hard work and academic success. This could be especially true for young people from families with few or no resources. The feeling that a lot is at stake may be stronger. I personally think all of this sounds plausible. However, I still want to be clear that this study did not test these mechanisms.
There are probably also even more negative explanations possible. Persevering more, for example, may also go hand in hand with greater pressure, greater fear of social decline, or the belief that school and, by extension, education is one of the only ways to escape a precarious economic situation. That is also why the authors explicitly warn that their findings do not mean that economic inequality is ‘a good thing’. It could just as well be a form of adaptation to an unfavourable environment.
Another point to keep in mind is that perseverance in the PISA study is measured solely through self-reports. This means we have no idea how long the students persevere in completing the assigned task at school. The second thing to notice is that there were changes in how perseverance was measured from one PISA round to another!
And so I have to close this post, still stuck with a lot of questions after reading this study. Not because the research is bad. I really don’t think it is. The researchers have found a remarkable association. This association proves quite robust across four PISA rounds and across a range of alternative analyses. But that is precisely why I want to know what is behind it.