While everybody seems to be looking for the holy grail in education – yes, Elon Musk, I’m looking at you – sometimes from little things, big things may come. A new randomized controlled trial by Rohher et al that Daniel Willingham shared past Friday is such an example. The subject: interleaving in mathematics. The study: a preregistered, cluster randomized controlled trial and in real life circumstances. The results? Pretty remarkable.

Some excerpts:

The study had two objectives. The first was to assess the efficacy of interleaved practice under naturalistic conditions. Most previous studies of interleaved practice have found that a greater emphasis on interleaving improved test scores, as we summarize further below, but these studies used small samples and some ecologically invalid procedures (e.g., laboratory settings or only one session of practice). The present study examined interleaved practice in a large number of classes at multiple schools over a period of five months, and all instruction was delivered solely by teachers who had no prior association with the intervention or the authors. These kinds of realistic conditions are important because promising interventions often fizzle in the classroom.

What was tested: blocked versus interleaved

A typical mathematics assignment consists of a group of problems devoted to one skill or concept. For instance, a lesson on slope is usually followed by a set of a dozen or more slope problems, and this format is called blocked practice

…

In an alternative approach known as interleaved practice, problems within an assignment are arranged so that no two consecutive problems require the same strategy, where strategy is defined loosely to include a procedure, formula, or concept. For example, a slope problem might follow a volume problem, and a probability problem about independent events might follow one about dependent events.

How big was the group of students being used in the study?

15 teachers and 54 classes.

The results?

In the large-scale randomized control trial presented here, a higher dose of interleaved practice increased scores on a delayed, unannounced test. The effect size was large, and a positive effect was found for each of the 15 teachers. This finding is consistent with the results of previous small-scale studies of interleaved mathematics that found test benefits with a variety of materials, procedures, and students. Taken as a whole, the extant evidence suggests that interleaved mathematics practice is effective and robust, although we list several caveats below.

…

(One month after the blocked versus interleaved classes…) …students took an unannounced test, and the interleaved group outscored the blocked group, 61% versus 38%, d 0.83.

Wait that is a very big effect?

The effect size observed in the present study might seem surprisingly large for a classroom-based experiment, but this might be due to the fact that interleaved mathematics practice combines three potent learning strategies, as explained in the introduction. First, the mixture of different kinds of problems within each assignment provides students with an opportunity to practice choosing a strategy on the basis of the problem itself, which is precisely what students must do when they encounter a problem on a cumulative exam or other high-stakes test. Second, interleaved mathematics practice inherently ensures a greater degree of spaced practice of each particular skill or concept across assignments, allowing students to exploit the spacing effect. Third, interleaving might encourage students to engage in the strategy known as retrieval practice by leading them to recall, or at least try to recall, the information needed to solve the problem (e.g., slope rise/ run). The secondary benefits of spacing and retrieval practice are not trivial. In one commissioned evaluation of 10 learning strategies, spacing and retrieval practice were the only strategies to receive the highest possible rating

Oh, and btw about those little things and big dreams?

While many unproven and expensive educational products continue to garner media attention and tax dollars, numerous classroom-based randomized experiments have found benefits of straightforward interventions requiring neither technology nor proprietary materials.

No comment.

Abstract of the study:

We report the results of a preregistered, cluster randomized controlled trial of a mathematics learning intervention known as interleaved practice. Whereas most mathematics assignments consist of a block of problems devoted to the same skill or concept, an interleaved assignment is arranged so that no 2 consecutive problems require the same strategy. Previous small-scale studies found that practice assignments with a greater proportion of interleaved practice produced higher test scores. In the present study, we assessed the efficacy and feasibility of interleaved practice in a naturalistic setting with a large, diverse sample. Each of 54 7th-grade mathematics classes periodically completed interleaved or blocked assignments over a period of 4 months, and then both groups completed an interleaved review assignment. One month later, students took an unannounced test, and the interleaved group outscored the blocked group, 61% versus 38%, d 0.83. Teachers were able to implement the intervention without training, and they later expressed support for interleaved practice in an anonymous survey they completed before they knew the results of the study. Although important caveats remain, the results suggest that interleaved mathematics practice is effective and feasible.