A character can be afraid, or angry, maybe sad. You can read this clearly in the children’s book. But if you look at the picture on the same page… There is a good chance you will see someone who looks… strikingly calm or happy. That is the finding of a new NBER working paper by Anjali Adukia and colleagues. The researchers examined how emotions are represented in both American school textbooks and award-winning children’s books. In short, the topic is the congruence between words and images.
That is quite different from what I initially thought the study was about. The title of the study is (How) Do We Teach Emotions?, but strictly speaking, this study didn’t examine this. The authors did not test any children. So we don’t know whether this pattern influences how children recognise, understand, or even express emotions. What they did investigate is the emotional world that books present to children. And this world turns out to be surprisingly different depending on whether you read or look.
On the one hand, the researchers analysed school textbooks from California and Texas from the past three decades, including books for language arts, science, and social studies. In addition, they also examined a century (!) of award-winning American children’s books. Using AI and several different methods, they mapped eight emotions: anger, calmness, confusion, disgust, fear, happiness, sadness, and surprise.
When looking at the texts, they find a fairly broad palette. Happiness is the most common emotion at around 29 per cent, followed by sadness at around 20 per cent. But anger, fear, confusion, disgust, calmness, and surprise also appear regularly. When they looked at the illustrations, however, the picture changed… well, literally (yes, pun intended).
Just two emotions made up around 90% of the emotions the researchers identified in images: calmness and happiness. This difference also appeared to be fairly stable across time and contexts. It appeared in textbooks and children’s books, in material from California and Texas, and across different periods. Even when the researchers examined different groups of characters, the same general pattern was found.
Now, you might think that illustrators simply choose the happier moments in a story to depict. After all, a book can be about sadness without every illustration having to show a sad face. That is why the researchers deliberately also compared texts and images on the same page. And then the phenomenon becomes even clearer!
When the text describes anger, only 13 per cent of the accompanying images also show anger. For fear, this is 9 per cent, for sadness 10 per cent, and for surprise 8 per cent. Only for calmness is there a strong correspondence between text and image. Well, I wonder why that might be? When the text describes an emotion other than calmness, the image still shows a calm face in 58-67% of cases.
On almost half of the pages examined, the researchers even find no overlap at all between the emotions in the text and those in the image. They then also checked whether this might be because the text and illustration concern different characters. Even when they matched characters as closely as possible, this striking pattern remained.
So what do children learn when stories tell them that characters are afraid, angry, or sad, while the faces they see alongside those stories remain mostly calm and happy? The authors suggest that this contrast may teach children an implicit norm. You can experience all kinds of emotions. But you are expected to remain calm or happy on the outside. They support this idea by citing prior research linking exposure to emotionally rich content to emotional vocabulary, perspective-taking, and emotion recognition.
But this is where, in my view, the researchers make quite a leap, a bigger one than their data allow. This study does not show that children actually pick up this implicit message. It is a plausible hypothesis arising from the findings. But it was not an outcome measured in children in this study.
There is another interesting possible explanation in the study. Perhaps it is not only about authors, publishers, and illustrators. The researchers linked some of the books to data on purchases by American households and to public library collections. Here, they find evidence that books with traditionally more ‘negative’ emotions on their covers are chosen less often. My own children’s book doesn’t really have any emotions on the faces on the front cover. Mila looks pretty calm on the back cover. Phew.
In other words, perhaps to some extent we simply buy the children’s books we, as parents, want to buy. But here too, I would be cautious. This was not an experiment in which buyers were randomly shown different covers. The authors therefore point to a possible role of consumer demand, not to a proven causal mechanism.
As mentioned, in this study the scientists used generative AI for emotion recognition, including GPT-4o-mini for text and Gemini 2.5 Pro Preview for images. That might sound like a reason to take the results with a considerable grain of salt. It doesn’t have to be, as the researchers put quite a lot of work into checking this. They also repeated their analyses using other methods, e.g. a lexicon-based approach, a separate language model, and more traditional image recognition. When they did this, the exact percentages did shift, particularly because the distinction between calm and happy proved to be difficult. Still, the central pattern remained clear: a great deal of emotional diversity in the text, much less in the images.
They also compared the AI assessments with those of human raters, as they should. This showed that recognising emotions is anything but perfect, even for humans. That makes me reluctant to attach too much importance to every individual percentage. But the fact that different methods yield the same general pattern makes the main finding considerably more convincing.