Retrieval practice: Better to space it out, or keep it together?

Don't look now, but we're entering into the final one-third of Fall 2020 semester. As we do, we can expect the emotional and physical labor of teaching well to mount, right along with the grading load and the pressure that students experience. It's going to be a great help for us who are teaching to look for simple, research-supported ways to help students learn in these last few weeks of the term. Let's look at a research article from 2016 that gives us some insight on one such technique: the venerable but (in my view) under-utilized concept of retrieval practice.

Hopkins, R. F., Lyle, K. B., Hieb, J. L., & Ralston, P. A. S. (2016). Spaced Retrieval Practice Increases College Students' Short- and Long-Term Retention of Mathematics Knowledge. Educational Psychology Review, 28(4), 853–873. https://doi.org/10.1007/s10648-015-9349-8

Retrieval practice is simply "the act of actively trying to recall information". It is an idea almost as old as education itself. Almost every school kid has used flash cards, for instance, to study for a test or memorize factual information. Retrieval practice goes a bit further than this, by using active recall tasks not merely to memorize or to prepare for an assessment but as a way to get information into long-term memory and make the process of accessing that memory faster and easier. Retrieval practice can take many forms, from flash cards to quizzes on an LMS to clicker questions.

While not all learning is or should be memorization, all learning and intellectual work involves having certain concepts in long-term memory and bringing them forward easily, even unconsciously. In my discipline of mathematics, for example, we talk about mathematicians having "mathematical intuition" --- an ability to notice aspects of a problem and using those observations to have a new idea about it or craft a simple solution. Even beginning calculus students use that intution when they calculate a derivative, by remembering similar functions from their experiences that have the same overall structure. That intuition, as Nobel Laureate Herbert Simon once said, is "nothing more and nothing less than pattern recognition" --- which involves having a supple memory.

The brain science behind retrieval practice is solid, and there's a reason why it's featured in many well-regarded books on effective college teaching such as Small Teaching and Make it Stick. It's also a technique that's particularly well suited for online and hybrid courses thanks to simple online polling platforms (like I described in an earlier post about peer instruction). In my experience, students also enjoy doing it. So, what can we learn about how to do it better?

What this study is about

In this study, the researchers looked at massed versus spaced retrieval practice. The former is when someone does retrieval practice in a short time window following the initial acquisition of the concepts. The latter is when the retrieval practice is more spread out in time. For example, in a flipped learning environment, something I commonly do at the beginning of a class meeting is give a three-question poll asking students to bring to mind some of the main ideas of their pre-class work. If the only (or primary) time I quiz students over that material is at that moment, then this would be "massed" practice. But if I changed up those polls to include two questions from that day's pre-class work and one question from the pre-class work of two weeks ago, I am "spacing" the retrieval practice.

It's not obvious that one strategy for retrieval practice should be better than the other. On the one hand, massed practice will "strike while the iron is hot", dealing with the concepts while they are still fresh. On the other, spaced practice forces students to really pull things from memory and to review, which often has a more salutary effect on learning. So the researchers here looked at three main questions:

1. Do learners retain more information from spaced retrieval practice than they do from massed retrieval practice?
2. Do learners with some exposure to spaced retrieval practice but also with some exposure to massed retrieval practice, retain more of what they learned than students who experience only massed retrieval practice?
3. Do students retain more of what they learn from massed retrieval practice if they also have some exposure to spaced retrieval practice, compared to students who only experienced massed retrieval practice?

The first two questions are pretty straightforward. The third one is more subtle. The researchers here wanted to see not only if spaced retrieval practice produced better results in terms of memory and recall than massed practice; they also wanted to see if spaced practice improved student's ability to get the most out of massed practice.

How the study was designed

The study was conducted with students in an "Introductory Calculus for Engineers" course, which was designed around a number of learning objectives --- 214 objectives to be precise. Students were quizzed over a subset of those objectives over 6 course units, using problem-solving quizzes that involved retrieval of "objective-critical information". Students were assigned into a control or experiment group, both taught by the same instructor. (It wasn't truly random assignment, as the groups were first manipulated for equivalency in racial and gender composition as well as ACT math and GPA scores.) Both groups received two quizzes per instructional unit, times 6 units. The researchers also looked at questions related to these learning objectives that appeared on the course's final exam.

Students in the control group received quizzes in the first four units of the course with three questions that covered each of six target objectives, all of which had been recently introduced (so it was massed retrieval practice). Student in the experimental group also got those quizzes except half of the questions were not recently introduced but were from earlier material. Notice that this means the group structures were not simply the control group getting massed practice and the experimental group getting spaced practice: The control group got only massed practice but the experimental group got some massed and some spaced practice.

This allowed the researchers to study that third research question. They were able not only to compare massed versus spaced practice between subjects (straight-up control vs. experiement) and within subjects (the experiemental group's performance on massed practice vs. the experiemental group's performance with spaced practice) but also the experimental group's massed practice vs. the control group's massed practice.  (Direct link for image)

What the study found

The researchers found that:

• Within the subjects in the experimental group, performance on spaced objectives was statistically significantly better than performance on the massed-practice objectives;
• Between the subjects in the experiemental and control groups, performance on the spaced objectives by the experimental group was statistically significatly better than performance on the same objectives done via massed practice by students in the control group; and
• Also between subjects, when looking at the learning objectives where both the experimental and control groups experienced massed practice, the performance by the experimental group was higher --- but the difference not statistically signifcant. It just missed the cutoff for significance with a p-value of 0.058. (Direct link to image)

The latter result would indicate that there's something to the idea of "doses" of spaced practice being related to overall improvement in massed practice; but no conclusions can be drawn about it yet. This might be an area for further research.

A very nice feature of this study, too, is that the researchers followed these students into the subsequent course in their curriculum, called "Engineering Analysis I". They looked at the first unit exam in this course, which was largely review of content from the Introductory Calculus course; the third unit exam, which had nothing from the calculus course; and the final exam. This allowed the researchers to do a kind of mini-longitudinal study to see if student's knowledge, especially their memories, of the calculus material persisted into the future. (Direct link to image)

They found that students from the experimental group in the calculus course scored higher on the first unit exam than students from the control group, although again the difference was not statistically significant, with a p-value of 0.078. On the third exam, however, the scores between the groups were almost identical; and on the final exam the students from the experimental group did score significantly higher than students from the control group.

What this means for the rest of us

I think one of the foremost lessons from this study is that retrieval practice really works for helping students succeed the fundamental step of committing concepts and information to long-term memory and problem-solving with it later, and that spacing out the practice is generally better than doing it all at once.

We also learn here that spaced practice might actually help students get better in any form of retrieval practice. Although the stats here don't let us conclude this with a high level of confidence, it's an intriguing notion that is borne out at least in part by the study's results. It also looks like the improvements in learning provided by spaced retrieval practice can persist into the future (although again, take this with some skepticism) so it's not just something we do in class to pass the time --- it can help students both now and later.

One takeaway from this study for me is that we should really be including spaced retrieval practice in every class meeting. I first encountered this study back at the beginning of the semester, and it was the main reason I elected to use polling questions in such a central role in my classes. Many students don't realize that frequent no-stakes quizzing is one of the best ways to learn, opting instead for low-quality methods like re-reading the material or simply staring at lecture notes. Part of our instruction of students can be to expose them to methods like retrieval practice and encouraging them to make it part of their daily regimen.

In fact, right now is a perfect time to be doing spaced retrieval practice in our classes. We're in the final 1/3 of the term, and so summative course-wide assessments are starting to loom large. Dedicating 5-10 minutes per class meeting for simple polling questions --- using free software like PollEverywhere, or the polling feature in Zoom, or just with people holding up index cards with the responses to a multiple choice question on them --- and continuing for the next 4-5 weeks will help students' memories get strong and will give the students legitimate confidence. This costs nothing, requires no special training or technology, and can be implemented tomorrow.

Where does that 5-10 minutes come from? It can come from lecturing less; or planning things our better so there is less wasted time; or from cutting out some activity that has lower instructional value than retrieval practice or which doesn't align well with our learning objectives and could be relocated to an online asynchronous format. I'd argue that spaced retrieval practice is one of those instructional practices that really maximizes the quality of synchronous or face-to-face time, and as such it should get priority over lesser practices.

Further reading

Whenever I read a research paper, it generates more stuff for my own reading list. This time three articles stood out which are going in the queue. I encourage you to check these out too, and join me in carving out three pomodoros per week for research reading.

• Rowland, C. A. (2014). The effect of testing versus restudy on retention: a meta-analytic review of the testing effect. Psychological Bulletin, 140, 1432-1463.
• Rohrer, D. (2009). The effects of spacing and mixing practice problems. Journal for Research in Mathematics Education, 40,4-17.
• Mayfield, K. H., & Chase, P. N. (2002). The effects of cumulative practice on mathematics problem solving. Journal of Applied Behavior Analysis, 35,105-123.