Ah, the replication crisis. One of my favourite topics. And yes, here is another example. This one involves beer.

In 1995, a study appeared that was almost too good not to cite. It looked at waiters and bartenders at Oktoberfest, who were said to perform worse on a simple physics task than people without experience carrying beer glasses.

The task is classic. Draw the water level in a tilted glass. The correct answer is horizontal relative to the ground. Yet many people make the mistake of tilting the glass and the water together.

Hecht and Proffitt reported something intriguing. People who work with liquids professionally would make that mistake more often. Their explanation was elegant. Servers develop an object-based frame of reference. They focus on the rim of the glass to avoid spilling. That habit then interferes with performance on a more abstract task. Expertise as a liability.

The story made sense cognitively as it fits a broader idea that experience can lead to rigidity. The study was widely cited as an example of how expertise can undermine performance. It appeared in management, marketing and cognitive psychology. It had everything a strong academic story needs. A paradox and a vivid real-world setting.

However, another study two years later by Vasta and colleagues found the opposite. It received far less attention, at least according to Google Scholar. Both studies also relied on small samples.

So, 30 years later, Tenney and colleagues returned to Germany. Back to the field. Waiters. Bartenders. Students. Bus drivers. More than four hundred participants. Preregistration. Open data. Everything as it should be.

And what did they find?

No effect.

Participants with experience did not perform worse on the water level task. The difference was essentially zero. If anything, the direction was slightly reversed compared to the original finding.

On a second intuitive physics task involving the trajectory of a falling object, there appeared to be an initial difference. However, that disappeared after controlling for demographic variables. So again, there is no solid basis for a striking expertise story.

What makes this case interesting, then, is not just that a classic finding does not replicate. It is also how easily an elegant first result takes on a life of its own. One study becomes an illustration. That illustration becomes a citation. And that citation becomes a standard example in talks and textbooks.

In education and psychology, this happens more often than we might like to admit. In practice, a neat experiment with a strong narrative punch becomes shorthand for a much more complex reality. Years later, when someone takes the effort to test it again more rigorously, the story turns out to be less robust than expected. Or, occasionally, it ends up as an entry in a book of myths.

At Oktoberfest, the beer still stays level. At least some things remain stable.

Image: https://nl.wikipedia.org/wiki/Bestand:Hacker-Pschorr_Oktoberfest_Girl_Remix.jpg