The advice to ask questions while students watch an instructional video is hardly surprising. But when is the best time to ask those questions? Before the video, so that students may have a better idea of what to pay attention to? Or afterwards, so that they have to retrieve the information from memory? A new study by Bart Schutte and colleagues examined exactly this question. And there doesn’t seem to be a clear winner.

The study involved 223 Dutch students in the second year of secondary education. They attended two schools and were in HAVO or VWO, the two higher tracks in Dutch secondary education. During a regular biology lesson, they watched an instructional video on genetics that lasted just over eleven minutes.

The researchers divided the students into five groups. One group answered seven multiple-choice questions before the video, and another answered them afterwards. In both cases, some students also saw the correct answer immediately after each question. A fifth group watched the video without answering any questions. Neat.

After the video, students did a short distractor task and then answered 14 test questions. They had seen seven of these questions before; the other seven were new. That distinction turned out to matter.

At first sight, asking questions helped. Students who answered questions before or after the video performed better on average than those who simply watched it. But if you look separately at the old and new questions, some of that nice result disappears. Students did better only on the information they had actually been asked about in the quiz before or after. For the rest of the video, asking questions made no difference. So asking questions did not seem to help students learn the entire video. They mainly remembered what they were directly tested on.

But there is more. Without feedback, which in this study took a very limited form, neither pre- nor post-video questions led to significantly better results than receiving no questions at all. When students were shown the correct answer straightaway after responding, both groups clearly outperformed the control group.

Feedback mainly mattered when students got an answer wrong. When they already knew the correct answer, showing it to them again didn’t help. Not very surprising, perhaps. Still, it suggests that asking a question alone may not be enough. What happens after a wrong answer matters too.

So, back to the question I started this post with. Should you quiz before or after the video? There is no clear winner. On average, neither is better or worse. It got more complicated when the researchers looked at whether students answered the quiz questions correctly. After a correct answer, post-video questions worked better. After a wrong answer, questions before the video did better. And after a wrong answer, feedback helped in both cases. But you can’t always predict the quality of their answers.

The researchers also looked at self-regulated learning. This didn’t make the story any simpler. Students who answered questions were less accurate at estimating how much they had learned. Even giving feedback made them slightly more likely to overestimate their performance. I didn’t expect that. On the other hand, students who answered questions were better at deciding whether to watch the video again.

As always, there are some limitations. In this case, the 223 students came from only two schools, and all were in HAVO or VWO, the more academically oriented tracks in Dutch secondary education. Also, the researchers tested what students learned almost immediately. Only a short distractor task separated the experiment from the final test. What happens after a day, let alone a week? This study doesn’t tell us.

There is another caveat. Seven questions appeared in both the initial quiz and the final test. One way the intervention might increase test performance is by re-exposing students to the exact information.

What conclusions could we draw from this research then? Adding questions in an instructional video depends on what question you ask. The questions in this study produced results only for that particular information. Providing the correct answers after students gave the wrong ones seemed helpful. But should the intervention take place before or after a video? Again, this study show no preference. That could be a good thing: you can vary how you approach it in class!