Giving marks that indicate progress

This article originally appeared earlier this week at Grading for Growth, a blog about alternative grading practices that I co-author with my colleague David Clark. I post there every other Monday (David does the other Mondays). Check the end of this post for some extra thoughts that don't appear in the original.

Click here to subscribe and get Grading for Growth in your email inbox, free, every Monday.

This is the third installment of a series of posts focusing on the Four Pillars of Alternative Grading:

The Four Pillars are a model that seeks to identify the elements that alternative forms of grading have in common --- elements whose primary goals are improving grading and making it focused on student growth. In a previous post I wrote about Clearly Defined Standards, and two weeks ago I wrote about Helpful Feedback. This week I'm on to the third pillar: Marks Indicate Progress.

However, before talking about how marks can (and should) indicate progress, we’ll start with a couple of important things: What are marks at all, and why is there a problem with the traditional way of using marks?

What is a mark?

Originally, this pillar was called Grades Indicate Progress. But the more David and I have written about the subject of grading, the more we believe there needs to be a distinction between the process of grading and the way we communicate and record the results of grading, as well as a distinction between grades in a course and grades on an assignment. These distinctions aren’t easy to make, so let's unpack them.

Imagine a student, Alice, completing an essay for a class. Alice will, usually, submit that essay for grading. In a traditionally-graded course, Alice would get two things in return: An copy of her essay containing instructor feedback in the form of words, annotations, and so on; and a "grade" on the essay, perhaps a letter grade like "B-" or a numerical score obtained using a rubric.

For our purposes, the mark is the second item: The number, symbol, or string that sums up the results of a piece of work when measured against a standard.

American English usage doesn't typically distinguish between "marks" and "grades". In an American class, Alice would say that her essay got a grade of B- or 40/50. In fact, most American students wouldn't know what you meant if you used the word "mark" in reference to grades. But in other cultures, there are some differences1:

• A "mark" typically refers to the outcome of a single piece of work, whereas a "grade" refers to the outcome of an entire course. So Alice might receive a mark of B- on her essay, but a grade of A- in the class. Marks collectively lead to a final grade.
• "Marks" and "points" are often (but not always) synonyms. So if Alice and Bob are classmates, they would know that there were 50 marks available on the essay, and Alice might ask Bob, "I got a mark of 40, what did you get?"
• Therefore, as this guide from Cambridge University puts it, "Grading is the process of turning a numerical mark into some form of grade". The word "numerical" is in the original.

To possibly oversimplify it, a mark is what you put in a gradebook once a student's work is evaluated (you marked the work); whereas a grade is what goes on a student's semester report card. Both grades and marks are shorthand indicators of some level of achievement; the mark is given at the level of an individual piece of student work.

Some problems with traditional marking

In traditional systems, the mark earned on a piece of work is typically either a number, a score given as a percentage or ratio, or a letter grade, indicating the general category of quality that the work falls into. There are several critical problems with this approach to marking.

The biggest and most central problem is that marks are often used as an implicit form of feedback. But as feedback, number- or letter-based marks are not helpful. As I wrote in my previous post, the primary purpose of feedback is iteration. In order to be helpful, feedback should convey information back to the learner about their work that will be useful in crafting a next iteration of that work.

But a number or a letter grade just doesn't help in this way. Consider Alice again, who earned a 40/50 or B- mark on her essay. Assuming that Alice will be able to revise and resubmit that essay (a big "if"; we'll think more about this in two weeks), what information does the mark convey about her work? What went well? What issues did she have? Did she have a single major issue, or a bunch of smaller ones that added up? Is a 40/50 bad --- does it even warrant revision in the instructor's view? Or is it already "good enough," and further revisions would just be taking a good essay and making it amazing --- which may or may not be what Alice wants to do?

There's just not much that Alice can pull out of that 40/50 or B- mark that will be useful for improvement. It raises more questions than answers, and Alice is reduced to guesswork if she wants to grow.

But no worries, you might say, because the instructor surely has also left lots of feedback on the essay itself in the form of notes, suggestions, annotations, and so on; so Alice can just focus on that. But there are at least a couple of problems with that idea. First, instructors might be more incentivized to give marks than they are to give feedback; we have to give marks on student work, but feedback in traditional systems is neither required nor particularly encouraged. Second, the classic paper from Butler and Nisan (and its many follow-up studies) showed that students, when given both marks and feedback on their work, showed no statistically significant differences in either performance or motivation than those who just got marks. This definitely meshes with my own experience: If you give number grades and feedback, students will tend to focus laser-like on the marks and ignore the feedback.

The problems with points, in particular, don’t stop there. Statistical computations using points — which is how we traditionally convert marks into course grades — can hide key elements of student growth. For example, if that 40/50 Alice earned on her essay was followed up with scores of 48/50 and 50/50, then her average would be 46/50. But if her classmate Bob had scores of 50/50, 44/50, and 44/50 in that order, then his average is also a 46/50 — even though Alice has gotten better at essay writing over time, if we believe the numbers, whereas Bob appears to have either gotten worse or stagnated.

In fact, although points are numbers, we typically use them not as numerical data, where mathematical calculations such as averages have real meaning, but as categorical data — that is, as labels that represent our professional judgment on the student’s work. At best, they are ordinal data where we can say that a score of 45/50 represents “better” work than a score of 40/50. But performing mathematical computations on categorical data is like averaging ZIP codes, which are also numbers but not numerical data. We’ll return to this concept in a later post.

How to give marks that indicate progress

So how might we use marks in a better way — one that assists our helpful feedback and helps students grow? The point of our third pillar is to specify how to make them useful labels: By using marks to indicate progress.

Perhaps the best example for our audience is the process of submitting a paper to a scholarly journal. Once the article is reviewed, we get two things back: A lot of helpful feedback (aside from Reviewer 2), and a label: Accepted, Minor revision needed, Major revision needed, and so on. The label there is the "mark". And unlike number or letter grades, it actually indicates progress. It's not particularly helpful as feedback, because if you just got the mark and not the reviewer feedback, you'd be left guessing as to what to do next. But, it briefly sums up the outcome of your work and invites you to engage with the feedback the reviewers left.

Here's another example. My son took a swim class a few years ago, and here is the "report card" that he got:

As you can see, he got a mark --- "Level 5", which is sort of numerical, but the number was just a label that pointed to how much of a particular skill set in swimming he had attained. The mark indicates progress. I can see from his mark that he can do certain things well (swimming 30 feet on his front without a float, and his favorite, jumping off the 1 meter board) but other things he still needs to work on (like swimming 25 yards on his back). The instructor during the class gave him feedback on all of these things; the mark was not on the same level as that feedback, but it indicates his progress toward a goal.

I aspire to this same level of informativeness, student-centeredness, and simplicity every time I compose a grading system for a new class. Here's what I've learned about how to do it.

First, you do not have to give marks at all. This is the concept behind ungrading, which both David and I have tried and have written about (here’s David’s article, here’s mine). In fact because of Butler and Nisan, there is a good argument to be made for decoupling marks from feedback and then throwing out the marks, basing the course grade just on feedback, student portfolios, and self-evaluations. So interpret the third pillar as If you give marks at all, they should indicate progress.

Second, also realize that you need the first and second pillars in order for marks to work best. That is, marks can't indicate progress in the absence of clearly defined standards and helpful feedback. Without clearly defined standards, what do we even mean by "progress" --- progress toward what? And without feedback, as we've seen with the example of Alice or submitting papers to a journal, marks tend not to be very helpful. So although I've encouraged you to focus on just one pillar if that's all you can manage for now, realize that working on this pillar will require you to work on the previous two.

Third, use a small number of marks. Aside from ungrading, where there are no marks, you can use a marking system that has:

• Two marks (Pass/Fail). This is the number that Linda Nilson recommends for specifications grading. In specs grading, you set up clear standards or "specs" for what constitutes acceptable work, and you report back to students that their work either met the specs, or it didn't. The advantage here is simplicity and speed; it typically takes very little time to give just a thumbs-up/thumbs-down on a piece of work, time which you can then reinvest in giving Helpful Feedback and engaging with students. The mark indicates progress, although it's somewhat blunt; and it drives student attention to the feedback.
• Three marks. If you want more nuance than a Pass/Fail system, you can add a third level. This can be an additional top mark which you can use to indicate excellent or exemplary work, for example Excellent/Pass/Fail to distinguish work that meets the standards from work that exceeds them. Or, it can be an additional bottom mark that can indicate that the work is almost at the passing level. For example, many people use Pass/Minor Revision Needed/Fail. Here “Minor Revision Needed” indicates that the student has unlocked the right to complete a revision that addresses a few small issues and then earn “Pass” (rather than making a wholly new attempt).
• Four marks (EMRN). Or, you can go both directions and add an extra top level and an extra bottom level. This is the idea behind the EMRF rubric created by Rod Stutzman and Kim Race. I wrote about my initial use of the EMRF rubric in specs grading at my personal blog. Since then, I've modified it so the bottom level is no longer called "F" --- because students consistently interpreted this as "Failing" rather than "Fragmentary" --- and changed it to the EMRN rubric:

I think the EMRN/four-level approach is about as complex as you want to get. Once you start using five or more marks in your system, it gets complicated, and that violates our Prime Directive of alternative grading: Keep it simple.

Also, avoid loaded terms. For example, if you use a two-level system, we’d strongly advise not actually calling it "Pass/Fail" --- those terms are so emotionally loaded that they tend to shut students down rather than drive further engagement with the feedback loop. You can use some short string to represent those marks, like "Satisfactory/Needs revision" or "Meets standard/Not yet". (By our definition, the mark can be a short string like "Meets standards": It's a label, not necessarily a single symbol.)

Finally, as a corollary, I would advise using plain language like "Satisfactory" or "Needs revision" --- avoid trying to make your marks motivational, like "Awesome!" or "Jedi Master"; or making them excessively clever, for example emoji as marks. Apart from possible issues of cultural bias (not everybody knows Star Wars, or what the 😍 emoji means) you want the mark to indicate progress, not make the student feel good (or bad). Keep it simple and give the facts. If you want to give motivational messages to students (which is not a bad thing!) use the feedback for that, where you can communicate a lot more clearly.

Talking about marks seems boring and technical until you realize that they are the primary way that our students interface with the feedback loops that form the heart of our teaching. If we can do better with marking, then we have that much better of a chance to help our students grow.

Bonus extra thoughts

• My post from earlier this week contains some additional thinking about marks, and it sparked a lot of interest online. TL;DR – I'm doing ungrading right now with my Modern Algebra course, which (to oversimplify it) removes marks completely from the course, with students only getting a grade at the end (with lots of feedback instead). And I'm not sure the absence of marks is helpful to everyone, particularly people at the lower end of the ability/preparation spectrum, who might need marks as "signage" to help them navigate the course.
• My point here still stands – if you're going to give marks at all (and maybe you should?) then they have to be "good signage" — they should actually indicate progress, and there should be helpful feedback which the marks summarize and toward which the marks drive attention. Otherwise students are just going to look at the marks, ignore the feedback, and probably stress out.
• So, about my crack at using emojis as marks: I seriously considered it at one point, and my mention of it wasn't an April Fools joke like the one from last year that didn't land. I like the concept of abstracted symbols – with no connection at all to words or letters – being marks. But which emojis? 💩 seems like a good replacement for "Not Assessible" but it's a little on the harsh side.