Measuring educational quality is difficult. But that doesn’t make it pointless.

Education matters. Or more precisely: good teaching makes a difference. That’s been the core message of educational research, policy documents, and teacher training programs for years. However, understanding why measuring educational quality is so difficult reveals that differences often turn out to be smaller than hoped. Consulting the literature shows that studies on educational quality frequently report relatively small and, above all, inconsistent effects. This isn’t necessarily a problem in itself, but it does become problematic when we try to predict what works, design programs, or formulate policy based on those same effects.

In a new thematic issue of School Effectiveness and School Improvement, Charalambous, Praetorius, and colleagues focus on precisely this unease. In seven contributions, they expose why the impact of teaching on learning often seems so limited in empirical research, and what we can do about it. The ambition of the issue is clear: not just muddling through, but getting back to the fundamentals. Yet, it’s precisely there that it also rubs us the wrong way. Although the authors present themselves as critical voices in the debate, some assumptions remain that perhaps they deserve to be questioned more critically.

Take, for example, the central thesis that the problem lies primarily in the way we define and measure “educational quality.” There’s certainly something to that, and believe me: I could dwell on it for ages. Several authors rightly point out the vagueness and overlap of the concepts used. Without a shared conceptual framework, comparisons become difficult, and misunderstandings accumulate. The measurement problems are also real: observations can sometimes be selective, questionnaires are susceptible to interpretation, and what we consider “learning gain” varies greatly from study to study. White, Göllner, and Kleickmann use the lens model to demonstrate how far the reality of the classroom sometimes deviates from the scores in the tables. And Grützmacher and colleagues reveal that many studies rarely explicitly state what exactly they measure in students, let alone why that would be a good indicator of “effective” teaching.

But then often comes a leap. As if the problem can be solved entirely by better definitions, sharper theories, or more innovative models. With enough refinement, we will indeed arrive at clear conclusions. That seems overly optimistic to me. Because even in the most precise studies, learning often proves erratic, context-dependent, and challenging to predict. Fortunately, Vieluf, Proske, and Naumann do explicitly point this out. They break with the idea that teaching is a kind of “production process” in which good technique automatically leads to good learning results. Education is not an assembly line, but a social process full of interpretation, interaction, and nuance. Perhaps the small effects aren’t the problem, but rather the symptom of an overly simplistic paradigm.

And yet, it remains difficult to truly distance ourselves from that paradigm. Even in this special issue, some authors continue to search for “better models.” At the same time, it might be more useful to accept that education is inherently complex and that some things can’t be neatly captured in effect sizes or causal loop diagrams. This isn’t a plea against research, but rather against reductionism. We can readily acknowledge that educational quality is important, without pretending we can measure it precisely. The special issue also includes strong voices (such as Vieluf et al. and Renkl & Endres) explicitly calling for a paradigm shift.

What can we, as teachers, trainers, or policymakers, take away from this? First, be cautious about drawing conclusions about what works. Just because an approach is effective somewhere doesn’t mean it will work equally well for you, in your classroom, with your students. Second, be mindful of what you measure. Don’t just focus on assessments or satisfaction, but ask yourself what you actually want to see – and why. And third, keep thinking in terms of interaction. A good lesson isn’t the sum of individual techniques, but emerges from the interplay between teacher, student, and content.

Does this mean we should stop researching teaching? Not at all. Quite the opposite. Research can still be a powerful source of inspiration, especially when it acknowledges nuance, context, and complexity. Evidence-informed practice doesn’t mean slavishly following “the average effect,” but rather drawing inspiration from insights gained elsewhere—and then carefully translating them into your own practice. This special issue helps us take a more realistic, critical, and simultaneously hopeful view of this, not by seeking the perfect model, but by asking better questions.

At the same time, this is certainly no excuse to simply attribute everything to ‘complexity’ or ‘context.’ The fact that we struggle to grasp educational quality doesn’t mean all ideas remain equally defensible. Some practices have been repeatedly debunked in research—consider learning styles, multiple intelligences, or the notion that direct instruction is “outdated.” We really don’t need to create new models or contextual analyses for that. Anyone who wants to work evidence-informed takes the current state of knowledge seriously. Not as an absolute truth, but as a guideline. And yes, that sometimes requires letting go of old beliefs. But that, too, is professional practice. Especially in a field as complex as education, we need clear insights and the courage to actually use them.

Perhaps that’s the core of what this special issue teaches us: research can’t offer us certainty, but it can provide direction. No recipes, only insights. And then it’s up to us—as teachers, as educators, as policymakers to take action, with an open mind and a critical eye.

Leave a Reply