While everybody seems to be looking for the holy grail in education – yes, Elon Musk, I’m looking at you – sometimes from little things, big things may come. A new randomized controlled trial by Rohher et al that Daniel Willingham shared past Friday is such an example. The subject: interleaving in mathematics. The study: a preregistered, cluster randomized controlled trial and in real life circumstances. The results? Pretty remarkable.

Some excerpts:

The study had two objectives. The first was to assess the efficacy of interleaved practice under naturalistic conditions. Most previous studies of interleaved practice have found that a greater emphasis on interleaving improved test scores, as we summarize further below, but these studies used small samples and some ecologically invalid procedures (e.g., laboratory settings or only one session of practice). The present study examined interleaved practice in a large number of classes at multiple schools over a period of five months, and all instruction was delivered solely by teachers who had no prior association with the intervention or the authors. These kinds of realistic conditions are important because promising interventions often fizzle in the classroom.

What was tested: blocked versus interleaved

A typical mathematics assignment consists of a group of problems devoted to one skill or concept. For instance, a lesson on slope is usually followed by a set of a dozen or more slope problems, and this format is called blocked practice

…

In an alternative approach known as interleaved practice, problems within an assignment are arranged so that no two consecutive problems require the same strategy, where strategy is defined loosely to include a procedure, formula, or concept. For example, a slope problem might follow a volume problem, and a probability problem about independent events might follow one about dependent events.

How big was the group of students being used in the study?

15 teachers and 54 classes.

The results?

In the large-scale randomized control trial presented here, a higher dose of interleaved practice increased scores on a delayed, unannounced test. The effect size was large, and a positive effect was found for each of the 15 teachers. This finding is consistent with the results of previous small-scale studies of interleaved mathematics that found test benefits with a variety of materials, procedures, and students. Taken as a whole, the extant evidence suggests that interleaved mathematics practice is effective and robust, although we list several caveats below.

…

(One month after the blocked versus interleaved classes…) …students took an unannounced test, and the interleaved group outscored the blocked group, 61% versus 38%, d 0.83.

Wait that is a very big effect?

The effect size observed in the present study might seem surprisingly large for a classroom-based experiment, but this might be due to the fact that interleaved mathematics practice combines three potent learning strategies, as explained in the introduction. First, the mixture of different kinds of problems within each assignment provides students with an opportunity to practice choosing a strategy on the basis of the problem itself, which is precisely what students must do when they encounter a problem on a cumulative exam or other high-stakes test. Second, interleaved mathematics practice inherently ensures a greater degree of spaced practice of each particular skill or concept across assignments, allowing students to exploit the spacing effect. Third, interleaving might encourage students to engage in the strategy known as retrieval practice by leading them to recall, or at least try to recall, the information needed to solve the problem (e.g., slope rise/ run). The secondary benefits of spacing and retrieval practice are not trivial. In one commissioned evaluation of 10 learning strategies, spacing and retrieval practice were the only strategies to receive the highest possible rating

Oh, and btw about those little things and big dreams?

While many unproven and expensive educational products continue to garner media attention and tax dollars, numerous classroom-based randomized experiments have found benefits of straightforward interventions requiring neither technology nor proprietary materials.

No comment.

Abstract of the study:

We report the results of a preregistered, cluster randomized controlled trial of a mathematics learning intervention known as interleaved practice. Whereas most mathematics assignments consist of a block of problems devoted to the same skill or concept, an interleaved assignment is arranged so that no 2 consecutive problems require the same strategy. Previous small-scale studies found that practice assignments with a greater proportion of interleaved practice produced higher test scores. In the present study, we assessed the efficacy and feasibility of interleaved practice in a naturalistic setting with a large, diverse sample. Each of 54 7th-grade mathematics classes periodically completed interleaved or blocked assignments over a period of 4 months, and then both groups completed an interleaved review assignment. One month later, students took an unannounced test, and the interleaved group outscored the blocked group, 61% versus 38%, d 0.83. Teachers were able to implement the intervention without training, and they later expressed support for interleaved practice in an anonymous survey they completed before they knew the results of the study. Although important caveats remain, the results suggest that interleaved mathematics practice is effective and feasible.

[…] A good deal of the podcast focuses on the importance of learning how to learn. I remember distinctly telling my mother how ridiculous I thought it was that I had to take school subjects in which I had no interest and that I wouldn’t use as an adult. She told me that while I might never use the content per se, I would use the skills required to learn that content. I was “learning how to learn.” Despite my irritation then, I can admit she was right now. And I have found myself using the same response with my son. Some of his frustration comes from having to read books, articles, etc. in which he has no interest, and I do empathize with his argument there – you get a lot more out of the content when it’s a subject you enjoy, but state tests and school life don’t always go our way. But another point of frustration for him is the requirement now for math students not only to provide the right answer, but to show multiple ways to get to that answer. While I also found that silly at the beginning (mainly because I can no longer even begin to help my 7th grader do his homework as I really have only one way of solving any given problem), it makes a whole lot of sense to me now. Math is now taught as an art, not just a science. It’s creative. This allows students to learn more strategies for solving problems, which in turns makes them capable of solving other problems they may never have seen before by applying a known strategy. (Here you can learn more about this powerful approach referred to, somewhat confusingly in my opinio… […]