I am writing this blog post in Paris. I participated in a UNESCO conference where we discussed education and research with various types of professionals. This means that several studies are also being mentioned. This meta-analysis by Ahmed Tlili and colleagues flew under my radar before but looks relevant to educational discussions. In the world of educational innovation, personalized learning has been hype for a while now – and I noticed it here in Paris among people mainly concerned with technology and learning. The idea is simple: adapt the learning process to the unique needs of each student, and the results will improve automatically. But how strong is the evidence that this approach actually works? This recent meta-analysis sheds light on this question, and the findings are intriguing, to say the least.

The study, which analyzed 19 studies with over 3,800 participants, shows that on average, personalized learning positively impacts learning performance. But definitely read on. The effect size was estimated at 0.45, which is a medium effect by academic standards and, therefore, not bad at all. However, the results are not as clear-cut as you might hope.

An important nuance is that not all forms of personalized learning appear to be equally effective. For example, systems that use prompts and feedback or diagnosis and suggestions did not have a significant effect on learning performance. This suggests that the way in which technology is deployed is crucial to its success. Something we often noticed in other applications as well. In addition, factors such as educational level, subject area and even geographical location appeared to influence effectiveness.

For example, the largest effects were seen in technology subjects and higher education, while lower education levels and mathematics benefited less from this approach. Interestingly, countries in Asia and Europe showed better results than North America, which may be due to cultural differences and technology implementation.

What do we learn from this? Technology can help, but it is not a panacea. Replacing teachers with smart systems does not seem to be the way forward. In fact, the study emphasizes that technology works best as a complement to human guidance. Developing a “collaborative intelligence”, in which humans and machines work together to improve education, seems more promising.

Personalized learning may certainly have potential, but its success, as always in education, depends heavily on how, where, and for whom it is deployed. It is a call to look not only at the technology itself but also at the broader context in which it is applied. This was precisely one of the common threads in the conference I attended.

Abstract of the research:

Limited studies have quantitatively summarized the effect of personalized learning systems on learning achievement. To address this research gap, this study conducts a systematic review and a meta-analysis of personalized learning systems, where 19 studies (N = 3822) were included. The obtained results revealed that personalized learning has an overall medium positive effect (g = 0.45) on learning achievement. They further revealed that personalized learning systems based on Prompts & feedbacks and Diagnosis & suggestions do not significantly impact learning achievement. Finally, the results revealed that educational level, subject domain and continent might impact the effect of personalized learning systems on learning achievement. The findings of this study can contribute to the literature by explaining how the effect of personalized learning systems might vary depending on different moderating variables.