A week in replication, some failed, some successful: Big Five, note taking and stereotype threats

The past week I read several replication studies with mixed results:

First, although direct replications (using methods from Mueller and Oppenheimer 2014) yielded longhand-superiority effects, the effects relevant to the encoding function of note-taking were small and did not reach conventional levels of significance. Such outcomes do not support strong recommendations about whether students should take notes longhand or by laptop in class. Second, based on their evidence, Mueller and Oppenheimer (2014) hypothesized that the higher word count and verbatim overlap for laptop groups were in part responsible for diminished performance for laptop users. However, when combining results across the direct replications, differences in word count and verbatim overlap were large, whereas differences in performance were small (and not statistically significant). Thus, differences in word count and verbatim overlap do not appear sufficient to produce performance differences between longhand and laptop groups.

The effects of gender stereotype threat on mathematical test performance in the classroom have been extensively studied in several cultural contexts. Theory predicts that stereotype threat lowers girls’ performance on mathematics tests, while leaving boys’ math performance unaffected. We conducted a large-scale stereotype threat experiment in Dutch high schools (N = 2064) to study the generalizability of the effect. In this registered report, we set out to replicate the overall effect among female high school students and to study four core theoretical moderators, namely domain identification, gender identification, math anxiety, and test difficulty. Among the girls, we found neither an overall effect of stereotype threat on math performance, nor any moderated stereotype threat effects. Most variance in math performance was explained by gender, domain identification, and math identification. We discuss several theoretical and statistical explanations for these findings. Our results are limited to the studied population (i.e. Dutch high school students, age 13–14) and the studied domain (mathematics).

In both examples of failed replication it’s not the case that we should abolish the theories as such straightaway. It’s rather that these new studies show that it’s more complicated than thought maybe due to regional and age differences (stereotype threat), maybe due to maybe differences in personal experiences (note taking).

1 Comment

Filed under Education, Psychology, Research, Review

One response to “A week in replication, some failed, some successful: Big Five, note taking and stereotype threats

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.