Maybe we have been underestimating the Flynn Effect for decades

The Flynn Effect is a fascinating phenomenon:

The Flynn effect refers to a secular increase in population intelligence quotient (IQ) observed throughout the 20th century. The changes were rapid, with measured intelligence typically increasing around three IQ points per decade. (source)

Do note this element of 3 IQ points per decade, because a new study by Gonthier and Gregoire argue that this is probably an underestimation of… 3 IQ points when you look at the items in the WAIS test rather than looking at the total amount:

  • Items can change in their difficulty or discrimination over time.
  • We tested the impact of these changes on Flynn effects in 3 versions of the WAIS.
  • Over half the items showed significant differential item functioning.
  • Differential item functioning biased the Flynn effect by about 3 IQ points per decade.
  • The Flynn effect should be studied at the item level to obtain unbiased estimates.

Abstract of the study:

The items of intelligence tests can demonstrate differential item functioning across different groups: cross-sample differences in item difficulty or discrimination, independently of any difference of ability. This is also true of comparisons over time: as the cultural context changes, items may increase or decrease in difficulty. This phenomenon is well-known, but its impact on estimates of the Flynn effect has not been systematically investigated. In the current study, we tested differential item functioning in a subset of 111 items common to consecutive versions of the French WAIS-R (1989), WAIS-III (1999) and/or WAIS-IV (2009), using the three normative samples (total N = 2979). Over half the items had significant differential functioning over time, generally becoming more difficult from one version to the next for the same level of ability. The magnitude of differential item functioning tended to be small for each item separately, but the cumulative effect over all items led to underestimating the Flynn effect by about 3 IQ points per decade, a bias close to the expected size of the effect itself. In this case, this bias substantially affected the conclusions, even creating an ersatz negative Flynn effect for the 1999–2009 period, when in fact ability increased (1989–1999) or stagnated (1999–2009) when accounting for differential item functioning. We recommend that studies of the Flynn effect systematically investigate the possibility of differential item functioning to obtain unbiased ability estimates.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.