Double the fun: Two-stage exams in a mastery grading course

This fall in Modern Algebra, I'm using mastery grading. I gave a full rundown of my mastery grading setup for Modern Algebra in this post and described a "T-shaped" experience I wanted students to have --- a deep understanding of foundational concepts coupled with a broad ability to connect those concepts to each other and to bigger ideas and applications. I'm assessing the foundations using what I call Foundations quizzes, and we have a Proof portfolio to gets students working in the creative space of connections and applications. Both Foundations quizzes and Proof Portfolio problems fit well in the mastery grading world, a fundamental pillar of which is that students should have the ability to revise and resubmit assignments until their work reaches an acceptable level of competency, rather than just being one-and-done.

However, when I was putting the course together in the summer, I realized something was missing --- namely the space between foundational concepts and high-level proofs and applications. I'm thinking of things like

• Compute $\gcd(5150, 90125)$ using the Euclidean Algorithm and then use the Extended Euclidean Algorithm to write this GCD as a linear combination of 5150 and 90125.
• Explain the steps of a written proof of the infinitude of primes, or fill in the blanks on a partial proof, but don't prove it from scratch.

Both of these occupy what I think of as the middle 1/3 of Bloom's Taxonomy --- "Application" and "Analysis" along with the top end of "Understanding" and the bottom end of "Evaluating". They require Foundational knowledge but do not directly assess it; they sometimes verge on proof construction but do not require it at the level of the proof portfolio. I introduced a timed test to gauge this knowledge, what I eventually called a Connections exam --- the name indicates it's specifically designed to probe the space in between the bottom 1/3 of Bloom (covered by the Foundations quizzes) and the top 1/3 (covered by the Proof Portfolio).

As I was putting the course together this summer, I began to wonder: What about revision and resubmission? How would this work with Connections? How do you make a timed test^[1] with a revision policy that gives students sufficient opportunities to improve, without causing a grading avalanche in the process?

Two-stage testing to the rescue

One of the articles I stumbled across during my sabbatical leave was this blog post by Carl Wieman on two-stage exams. I read it, and the paper attached to it, and knew instantly that this was the answer to my revision/resubmission issue.

The idea behind two-stage exams is simple:

1. In the first stage, students first work individually on an exam for a subset of the exam time.
2. Then there's a short time during which students hand in their work and transition into groups.
3. Then, in the second stage, students get a clean copy of the exam or a portion of the exam, and they work the rest of the time in groups to answer the exam questions again.

This idea isn't new, and it's a basic pillar of a lot of good teaching techniques like team-based learning. It struck me as something that would work well in the class on a number of levels, so knowing that I cannot leave well enough alone with my teaching, I decided to use it for Connections exams in Modern Algebra. Here's how they work. Note that my class meets Tuesdays and Thursdays in a 75-minute time block each day.

• Stage 1 of each exam is broken up into 4-5 sections, essentially one problem per section and each section focusing on one big concept.
• Students get 45 minutes to work through Stage 1. I have a 45-minute timer running on the projector to keep things on track.
• When the 45 minutes are up, I start a 5-minute timer, collect students' Stage 1, and then put students into randomly generated groups of 3 or 4. Each group is given one copy of Stage 2, which consists of a subset of the sections from Stage 1, possibly slightly modified if needed to make sense in a group setting and shorter time limit.
• When the 5 minutes are up, students work in their groups through Stage 2 and write up one single group report. They get 25 minutes for this, at which point the exam is over.

Mastery grading and two-stage exams

In mastery grading, we usually don't use points for assessing work, but instead give grades that describe the work relative to quality standards that we discuss in the syllabus and in class. Here's how this works for Connections:

• Each section (which remember is roughly one problem) is graded Excellent, Satisfactory, Progressing, or Incomplete --- both in Stage 1 and in Stage 2.
• For any section that appears in both Stages, students get to keep the higher of the two ratings. This way, students' work in groups won't bring down their individual work or vice versa. The exception to this rule is that if someone turns in work on a section that earns Incomplete (e.g., they leave a section blank otherwise skip big chunks of the section) then the highest they can earn on that section is Progressing, even if the group work is "Excellent". (So, students can't deliberately tank on a section in hopes of having the group work bail them out.)
• Then, the exam itself gets an overall grade of Excellent, Satisfactory, or Progressing depending on how the individual sections turned out:

To earn a grade of "B" or "C" in the course, students have to earn Satisfactory or Excellent on all three Connections exams. To earn an "A", at least one of those exams must be "Excellent".

Revision and reassessment

The two-stage approach makes revision more efficient, because in a way the group stage is the first round of revision. Students will have already made an attempt on all the sections; working in groups allows them to instantly revisit that work and pool their attempts with others. So before they even leave the testing session, they've already taken and revised their work once.

After the test is over, students are allowed one revision of any remaining section on the exam that isn't at the level they want. Students just look through their tests, pick the sections they want to redo, and then I will make up a new version of those sections with different content but the same concepts tested. This will be one new version for the entire class --- making up a new version for every single student just isn't feasible and it's why allowing unlimited retakes doesn't work for bigger tests where it might work for small quizzes. Then students work them out and submit them via Blackboard for regrading, and then the grade is final.

So this way, I limit the amount of regrading and making up new assessments to just one per exam, three times total during the semester, but students really get two chances at revision per test. It feels like getting two revisions for the price of one, which is perfect.

What I like/don't like/am not sure about regarding this approach

Last Thursday was our first Connections exam and the first time I'd ever done this. Here's Stage 1, and here's Stage 2.

I really liked the results. Students worked more or less as you would expect them to on a 45-minute timed exam. But when I got them into groups, magic happened --- students normally reticent were talking it up with their classmates, clever and creative solutions were being tried out, good and honest questions were being asked, and students were teaching each other mathematics. It was really the best group work I've seen in a long time and definitely the best collaboration we've had this semester.

I asked students to reflect on their experiences, and here are some of the things they said:

• I think the test went really well. I liked stage 2 of the test because it allowed me to revisit certain problems from stage 1 that gave me some trouble and discuss it with others..
• I felt very rushed through the individual portion, but felt that there was plenty of time for the group portion..
• I felt a little rushed for time on Stage 1, maybe one more fill in the blank proof would have evened it out a little more. I really liked how it was two stages though. That really helped me learn. Also knowing that I am able to do a revision gave me piece of mind.
• Start with the group part of the test first. (That's an interesting idea.)

Many of the other student takes on the exam just repeat these --- they were rushed on the individual part but felt fine on the group part. I don't think I have ever, in 21 years of teaching, heard a student say they had sufficient time on a timed individual exam no matter how many or how few questions are on it, so I take the comments about being rushed with a grain of salt. However, it's worth considering adding (say) 5 minutes to stage 1 and subtracting 5 from stage 2.

One thing I did not like about the two-stage approach is that the group stage can put certain students at a disadvantage, for example neurodiverse students who might get overloaded or distracted easily from the noise and bustle of Stage 2. (It did get quite loud in both sections.) I realized this issue belatedly, so I let any student choose to opt out of the group stage and spend that time working on Stage 1 in a quiet room. I had one student take me up on that offer, so I'm glad it dawned on me, but it's not an easy issue to route around.

One thing I am not sure about is how the revision process will play out, because I am still grading Stages 1 and 2. I am hopeful that the one revision will be enough; I wish I could open it up for more, but seriously, this is a lot of grading we're talking about and you can only take on so much before you lose the ability to give timely and meaningful feedback.

I'm also not sure how well this process works if you have a class that doesn't meet for 75 minutes. It seems like it doesn't scale down well --- for example in a 50 minute class, you'd be giving something like 30 minutes for stage 1 and then 15 minutes for stage 2, which just doesn't seem like enough time to assess anything more than 2-3 ideas.

I was very excited to come across the concept of two-stage tests, and overall I am very happy how it's working out so far. It keeps a layer of individual mastery on the testing environment while also injecting collaboration and peer-to-peer learning. This strikes me as one of those "small teaching" methods for doing something simple to get a lot of high-quality active learning going on in your class. I'm looking forward to seeing how it evolves for the next exam.