Building Calculus: The grading system

Something happened this past week that made this whole series I'm doing about the building of my Fall courses become a lot more real: July ended. Now that I'm writing down an "08" for the month, I'm reminded that all this building isn't just an abstracted academic exercise but will come into contact with real students in just a few weeks' time. So I'd better get on with it.

In the previous posts, we've looked at the choice of modality for my Calculus class in the Fall, the all-important process of determining learning objectives, and how the activities and most recently the assessments will take shape. We've come now to the point that might interest a lot of people: The grading system for the class. We have a lot to talk about here.

It's all pointless

When I first started teaching, it wasn't uncommon for me to start with the grading system when building a class. I'd start with the axiom that we'd have three exams and a final, plus some quizzes (maybe homework). If I felt daring, I might throw in a project or presentations.  And then determine the number of points for these, then declare that 90% of the total was an A, 80% was a B, and so on. Easy, right? That's why it's tempting to start there.

I think those of us who teach and who have graded using points – which is pretty much all of us – have always had some deep suspicions about point-based grading systems, which we repress, because examining those suspicions would require a comprehensive rethinking about... everything. Six years ago, 15 years into my teaching career, I finally put those feelings into writing, and decided that I'd had enough of the tyranny of points. And thus began an exploration of what we now call mastery grading that continues today, and in this article.

Very briefly, mastery grading is an umbrella term, including such practices as standards-based grading and specifications grading, that refers to grading practices that share some common characteristics:

• Typically, student work is not graded using points, but instead is evaluated relative to clearly-stated criteria that describe acceptable quality. The grade is assigned with a two-level rubric, not necessarily literally "Pass/Fail" but some binary determination of whether the work meets the criteria or doesn't. Sometimes more finely-graded rubrics are used.
• Since there are no points, there's no concept of partial credit. Instead, students get significant, helpful, actionable feedback from the professor that gets them thinking about how to improve.
• Finally, mastery grading builds in ways for students to take the feedback and revise and resubmit their work, in a feedback loop that continues until the work has met the criteria (or the student runs out of opportunities).

I really liked the way Tri-County Early College phrased it: "All assignments must be completed at a level of competency and are in-play as long as that takes (i.e. grades are never used as a punitive measure and zeros are never given)." [My emphasis]

That's a very low-resolution overview of the concept. There's much more at the Official Mastery Grading FAQ.

Mastery grading in Calculus

Here's a description of how each individual assessment is graded and how students can revise and resubmit their work, according to the principles above.

Checkpoints are take-home exams for demonstrating skill in the 24 Learning Targets of the course. Each Checkpoint contains one (multi-part) problem targeting exactly one Learning Target, for each Learning Target that has been discussed in the class up to that point. These are cumulative, with each Checkpoint containing not only new problems for recent Learning Targets but new versions of problems from previous Checkpoints. Since last week, I've written two sample Checkpoints to illustrate the idea: Here is the sample Checkpoint 1, which will be given in week 2 of the course; and here is the sample Checkpoint 2, which will be given one week later. (If you're wondering, here is an initial schedule of when each Checkpoint will be released and what new targets it will contain in addition to all the older ones.)

Each problem on a Checkpoint is graded simply "check" or "x" depending on whether the student work meets the criteria. The schedule linked above also has brief descriptions for each Learning Target about what students will be asked to do and what will constitute acceptable work. When a student submits work on a Checkpoint (via working the problems out on paper and then submitting a scanned PDF to the LMS), I'll look it over, determine whether it's a "check" or an "x", then put the grade in the LMS along with written feedback on the work. If the grade is an "x", the student can attempt a new version of that problem on the next Checkpoint (or use an alternative method, which I described last time).

Application/Extension Problems (AEPs) are problems sets used for demonstrating skill in applying basic knowledge. I have 8 of these planned and will start drafting them next week. These are more extensive and nuanced, so instead of "check/x" I use the EMRN rubric which I showed last time. This work is also submitted as a PDF on the LMS, and as with Checkpoints, I examine the work, assign the letter, then give helpful feedback. Then students can revise and resubmit any AEP that earned an M, R, or N, subject to the Two-Item-per-Week Rule (no more than two AEP submissions can be made in any given week) and the Revision of N grades Rule which states that work that receives an "N" (Not Assessible) requires spending a token (see below), in order to prevent students from submitting work that is highly flawed or incomplete on purpose just to get partial feedback.  Otherwise there are no limitations on revision and resubmission.

The Daily Prep and Followup Activities that bookend students' engagement with content are also graded check/x on the basis of completeness, effort, and deadline compliance. They are trivial to grade. However, unlike other assessments, there's no opportunity to revise or resubmit; they are meant to be done once and correctness isn't part of the criteria. I am OK with that since it's so easy to earn a check on them — just do something reasonable for each item and turn it in on time!

All of the other assessments in the course are graded with points, but with good reason. Online homework is done through WeBWorK, which uses points and there's nothing I can do about that; however the system does allow students to redo problems as many times as they want if the answer is wrong. And the miscellaneous Engagement Credit opportunities will be graded either 0 points or 1 point, but these are just labels indicating that the opportunity was either done or it wasn't. It's just clearer to talk with students about accumulating points for these than to say something like "accumulate at least 90 'acceptable' marks".

How the course grade is determined

I still have to give course grades of A, B, C, D, or F, so having determined how each individual assessment is graded, the next task is figuring out how to map student work onto these five letters and their plus/minus variants.

First, one more thing about Learning Targets: There are two levels of attainment with these. Earning a single "check" on a Learning Target earns you what I call Proficiency level with that target. Earning a second check on that Target — by working a second Checkpoint problem sufficiently well, or using an alternative method – takes you to the next level which I call Mastery.

Back to the course grade, I find it helpful to start with the "C" grade. A "C" is considered baseline competency in the course — the "C" student has demonstrated the minimum level skill to warrant allowing them to go on to other courses that use this one. What does that look like? Individual instructors will have different settings here, but for me, baseline competency means:

• The student has demonstrated Proficiency on all the Core learning targets and Mastery on at least half of them.
• The student has demonstrated Proficiency, if not Mastery, on about half of the non-Core or "Supplemental" Learning Targets.
• The student has done a significant but not overwhelming amount of acceptable work on applications.
• The student has done a fair amount of correct work on the online homework.
• The student has demonstrated reasonable engagement with the class.

If I could truthfully describe one of my students in these terms, I'd feel OK — if not supremely confident — that if they went on to Calculus 2 or Physics or something that requires my class, they could succeed if they work hard and get help when needed. That, to me, is what a "C" signifies. On the other hand if one of my students did not meet one of the above descriptions, I'd have reasonable doubt about whether they are "baseline competent" in the subject, and this would warrant a grade below C.

From this broad description, it's fairly easy to make specific criteria for earning a C in the class. A student earns a C, if they satisfy all of the following:

• Earn Proficiency on all 10 Core learning targets and Mastery on at least 5 of them.
• Earn Proficiency on 6 (out of 14) Supplemental Learning Targets.
• Earn either an E or an M on 5 (out of 8) AEPs.
• Earn at least 140 points (out of 192) on WeBWorK problems. (That's 73%.)
• Earn "check" on a total of 34 Daily Prep or Followup Activities. (There are 24 of each of these, and we are counting the total. So if a student struggles with the Daily Prep, they can make up for it by doing more Followups, and vice versa. Also, that's 71%.)
• Earn at least 60 (out of 100) engagement credits. (Daily Prep and Followup Activites earn one engagement credit per "check", so satisfying the previous bullet earns 39 out of those 60; the other 21 come from miscellaneous activities.)

Students have to satisfy all these requirements to earn a C. If they miss one, their grade will be at most a C-. This is jarring to students because they are used to having poor performance in one assessment "made up for" through good performance on another. But here, I require across-the-board competency. Even if a student earns Mastery on all 24 Learning Targets for example, but doesn't show competence on the AEP's, they will not earn a C.

To get the criteria for a grade of B, the idea is that it's everything needed to earn a C, plus extras in terms of quantity, quality, or both. Likewise for an A, the criteria are meet the requirements for a B, plus extras. Here's what I decided on for these requirements, as well as requirements for a grade of D:

"M+" means "either an M or an E".

For the D grade, I had to decide what a safety net would be for a student that doesn't show baseline competency, but also shows significant progress. I decided roughly speaking that if a student was "halfway to a C" then that's the requirements for a "D". Students who don't meet all the requirements for a D, earn an F.

Tokens, plusses, minuses

A common feature of mastery grading systems is the token, which is fake currency (think: Bitcoin but for grades) which students can spend to bend the rules of the course. In the Calculus class, every student starts the semester with 5 tokens. They can spend them using this menu, where everything costs one token:

• Attempt a second Learning Target in a given week through non-Checkpoint means
• Submit a third AEP (either revision or new submission) in a given week
• Revise an AEP graded "N"
• Extend the deadline on a Checkpoint by 12 hours (request must be submitted prior to the original deadline)
• Extend the deadline on a WeBWorK set by 24 hours (request must be submitted prior to the original deadline)
• Purchase 3 engagement credits

About plus/minus grades: The  table above is used to determine the "base grade" in the course, which is the A/B/C/D/F grade without plus or minus modifiers. If it were up to me, there would be no plus or minus grades because I think they complicate things unnecessarily. According to my superiors, though, I must have some system for giving plus/minus grades — but there are no mandates for how I do it. Here are my rules for this:

• A "plus" is added to the base grade if all requirements for a base grade are satisfied, and the Learning Target (both Core and Supplemental) or AEP requirement for the next level up is also satisfied; and the big-picture portion of the final exam is passed. (I mentioned the final exam last time.)
• A "minus" is added to the base grade above in any of the following cases: (1) All requirements for a base grade are satisfied except one, and that one is no more than two levels below the others; or (2) the student meets the minimum requirements for a base grade (i.e. none of the requirements for higher levels are met) and but does not pass the big-picture portion of the final exam or (3) the student meets the minimum requirements for a base grade but does not complete the Functions Bootcamp satisfactorily by Monday, September 14.

(The Functions Bootcamp is a special unit I am making to get everyone up to speed on mathematical functions.) Like I said, this complicates things and I'd prefer not to deal with it at all. But, since I do have to, I've settled on having a "plus" awarded for completing one grade level and "going above and beyond"; and a "minus" awarded for "almost" situations, or for not doing sufficiently good work on the final — and to use as a stick to get people to complete the Functions Bootcamp.

What I expect, and what I'm not sure about

This system is an iteration of the grading system I described here that I ended up liking a lot, for reasons I like mastery grading in general: It produces fewer false negatives (poor grades but good understanding of the material) and false positives (good grades, poor understanding) and helps alleviate stress and anxiety over grades because of the robust revision policy.

I've used systems like this before and I have a sense of what to expect. Namely, students will find it confusing at first and many will freak out over it, especially given the unsettled nature of this coming semester. It looks weird and hard and nonintuitive and definitely not like their other courses or previous schoolwork. What it takes to help students adapt to it is careful explanation and presentation in the beginning — not necessarily explaining every facet of the system all at once in the syllabus but giving time for student to absorb it — and especially, just living with this system and working with it. Generally my experience is that 2-3 weeks into the course, around about the first exam or Checkpoint, it clicks with students and they come to really appreciate it. It helps to engage in consistent marketing, explaining the benefits of "eventual mastery",  being able to redo almost anything in the course, and pointing out the fact that students' grades never go down during the semester as the result of an assessment.

I also expect that no matter what I do, some students will continue to dislike this system and prefer the old way, which they believe is clearer and simpler — despite the fact that points-based systems hide so much information about where the points come from that you can't really say they are either clear or simple. I just have to be ready to be patient with those folks, and accept that I'll never win some of them over.

In terms of grading load, my experience is that it's about the same or maybe a little less than traditional grading systems, despite the revisions and regrading, because each individual item goes from taking several minutes per student (Should this get 10 points out of 12? Or 8? or 9? or 6? or 0?) to seconds (Is the work good enough or not?) It's not distributed the same; the work is actually pretty light during the semester but ramps up tremendously at the end (thanks to procrastination). I'll have a lot of work to do in December. I've come to think of it like being a CPA during the US tax return season.

I will probably change this system before classes start. In particular I'm thinking about eliminating the whole engagement credit portion of the grade and just counting the Daily Prep + Followup number as a proxy for "engagement". It would simplify things; but I like having incentives for being engaged. Generally, my modus operandi is to write up a fully realized course, then tell myself to cut out 10% of it to simplify. I'm not sure what will get cut, but the cuts are coming.

I'm also not sure how students will respond to this system as a result of remote instruction. I've used a similar system in online and hybrid classes before, but never when students were forced to do online or hybrid and the world seemed to be spinning out of control. Will this system be just too much for students to handle? Will I need to simplify beyond my customary 10% cuts? I don't know. I'll have to return to this later once more of the course is built up and decide. And when classes start, I ask for frequent feedback, create some simple tools for tracking grades (like a checklist), pay close attention to students, and be ready to change things up.