Steps toward excellence: Connecting objectives and assessments with activity

As we enter into the second month of The Big Pivot, data are beginning to emerge about how students are handling it. One report, outlined in Inside Higher Ed, surveyed 513 current college freshmen, sophomores, and juniors and found that 50% of those surveyed said that their current online instruction was worse than the education they received face-to-face; 13% responded that the online instruction was "a lot worse". On the surface, it's not pretty.

On the other hand, I find it strangely encouraging, because that 63% of respondents who said their current online instruction is worse, are right. For the most part, it is worse, and what else would one expect? Most of what we're doing is a first-iteration hack, done under almost impossible conditions and with a minimum of training or even forethought. If you collected the same data from the faculty teaching these courses, I'm confident that the corresponding result would be north of 90%. The fact that 37% of our students are at least willing to reserve judgment about our online teaching right now tells me there's hope that if we can get better, we might be able to win them back.

It seems like a good goal for the next year to improve our online instruction so much that if we re-ran this survey in April 2021 these percentages would flip from 63/37 unfavorable/favorable to 37/63. As I've been saying in this series of posts so far, this is a tall order, but it's doable if we focus on the core elements of good online instruction and take the time and space we have to focus on building that core.

I wrote that Step 1 of this process is writing clear and measurable learning objectives. Step 2 is aligning assessments and grading systems with those learning objectives. I realized there's an intermediate step that I skipped because I thought it was so obvious it didn't need mentioning, but the more I look around, the more I see that it's not obvious at all.

What students actually do in the class

I'm going to call this Step 3, but it's really more like Step 1.5 because it deals with the link between learning objectives and assessments:

Step 3: Clearly connect the learning activities in the course to both the learning objectives and the assessments.

A "learning activity" is something that a student does as some incremental step toward attaining a learning objective. An assessment, if we follow Step 2 and align those with learning objectives, can be a learning activity. But this term also includes potentially unassessed activities like participating in a discussion, doing a reading assignment or a worksheet, or contributing to a group project.

What this step is telling us is that to the greatest extent possible, learning activities should provide practice toward eventually attaining one or more of the learning objectives. Any other activities that don't point students toward mastering learning objectives might be fun and interesting, but they are extraneous. And in an online course, we really can't afford "extraneous".

On a practical level, this means:

• Learning activities should be based on the learning objectives. At my university, one of the first questions we had to deal with after we pivoted online was simply, What are students supposed to do in our classes? Some of our faculty were sticking with synchronous delivery while others went asynchronous, but the question was the same. If you take time to really build the learning objectives, the answer is actually easy: Have students do whatever it is the learning objectives say that they should eventually be able to do. That is, don't just make up activities for the sake of activity; look to your learning objectives and have students work on whatever incremental step is needed in the moment to move toward mastering those.
• Learning activities should be clearly connected to the learning objectives. In the first post in this series I mentioned that clarity, structure, and predictability are key elements for student success in online courses, especially students with learning disabilities. Creating a visible, unmissable link between activities and objectives provides this coherence. You could simply state on an activity, "This activity provides practice with the following learning objectives..." and then list those. Or you could create a system of nomenclature, for example labeling the learning objectives for Module 5 in your course as objectives 5.1, 5.2, etc. and then labeling an activity aimed at objective 5.2 as "Activity 5.2.1". Or make a mind map that shows a visual connection between objectives and activities.
• Learning activities should be consistent with the assessments. This doesn't mean that assessments should simply replicate the activities; it means that if learning objectives and activities are aligned, and learning objectives and assessments are aligned, then (by the symmetric and transitive properties) the assessments and activities ought to be aligned. The assessments should not make a hard left turn away from the direction that the activities were heading, nor should the activities be irrelevant or extraneous based on the learning objectives. The path from learning objectives to activities to assessments ought to be as straight and continuous as possible.

Examples and non-examples

Here are some of the learning objectives from a module in Calculus that I presented in the first post of this series:

1. Correctly use the alternative $dy/dx$ and $d/dx$ notation to take and express derivatives.
2. Compute derivatives of constant, power, and exponential functions including the function $f(x) = e^x$.
3. State the Power Rule, Constant Multiple Rule, and the Sum Rule.
4. Apply the Power, Constant Multiple, and Sum Rules to compute derivatives of combinations of constant, power, and exponential functions, including polynomial functions.

Here are some possible activities that align with these learning objectives:

• For Objective 1: A multiple choice quiz on the course LMS with items that have students select the correct notation that corresponds to an English statement, like this:

(Addresses Objective 1) Which of the following is the correct notation for "the derivative of a function $y = f(t)$ with respect to $t$"?
(a) $dy/dx$
(b) $dx/dt$
(c) $dy/dt$
(d) $d/dt$
(e) $d/dx$
(f) $d/dy$
(g) None of these

That could be a clicker/polling question too.

• For Objectives 2-4, a handout that begins with a focusing activity asking students to write down the statements of all the rules mentioned in Objective 3; then asks students to work out several derivatives using those rules; then students pick one and record a 3-minute video of themselves working out the complete solution, and post the video to the class FlipGrid.

The possibilities for well-aligned activities are basically endless. All we have to do is make sure there's an obvious connection, made clear to the students, between the activities and the learning objectives. And if we align the assessments with the objectives, these activities create a sort of blueprint for those assessments. This makes the course more coherent to the student; and it saves the instructor a lot of time and effort later.

Here are some activities that are not as well-aligned:

• For Objective 1: Write a 500-word discussion board entry on the history of the $dy/dx$ notation for derivatives. Not that this isn't interesting, but its relevance to the learning objective is tenuous at best. Also, the Bloom's Taxonomy level is too high; Objective 1 is an Apply or Understand task, whereas an essay assignment is more Analyze or Evaluate. The concept of "alignment" refers not only to outcomes but also to the level of cognitive operation in the objective.
• For Objective 2: A multiple choice quiz where students have to select the correct derivative of a given function from among a list. Here, the Bloom level of the activity is too low for the learning objective, which asks students to compute something (Apply) whereas the activity is just asking students to select something (Understand).

Active activities are active

One major takeaway here is that active learning still needs to be at the heart of our instruction, even if it's online. If well-designed learning objectives are anchored to concrete action verbs, then activities that are aligned with those objectives will also revolve around action verbs --- that is, around doing things beyond just active listening and note-taking, that are being done so that students can build their own understanding of the concepts in the course. Just like in a face-to-face course, lecture can have its place and function very well in that place; but that place cannot be all places at all times, because quite simply, listening and note taking by themselves will not satisfy the learning objectives.

In an online setting, it's important that activities are not just active but interactive --- a purposeful back-and-forth between the student and the material or between the student and other human beings, preferably both. This will probably require rethinking some of the things we currently have students do. For example, reading a text, when done by an expert reader, is a highly interactive experience; however our students are mostly not expert readers in our disciplines, so some kind of activity should accompany the reading to guide the student through a properly interactive session with it.

A word of warning/advice: I suspect many of the 63% of students in that survey who preferred face-to-face instruction are getting online instruction that consists mainly of non-active work --- some combination of recorded lectures and very basic worksheets. Students would be right to want something more. But at the same time, my experience is that while students may want more, they are used to getting less (the lecture-and-worksheet approach), and introducing active learning to the online experience can be fraught with resistance and misunderstanding. Proceed with caution. Explain why you're doing what you're doing; make the learning objectives -> activities -> assessments pathway clear; and as always, communicate early and often with students about what they're experiencing and what they need.

Why does all of this alignment business make our online instruction better? Three reasons.

First, it makes our courses more coherent and therefore easier for online students to navigate and use. In a face-to-face setting we can admit a certain amount of noise in the signal with our instruction because we have the in-person environment to make up for it. In the online environment, that's much less the case, and so having less noise, less "fat" overall will help students to learn better --- especially the most vulnerable students.

Second, because alignment makes a course "leaner", it reduces work and cognitive load for everyone. Students can get the same or even better learning for a fraction of the cost; we expend less energy in grading and prepping because we're letting the learning objectives do a lot of the heavy lifting and make many of our decisions for us.

Third, if alignment makes a course simpler, more efficient, and more coherent, it frees us up to be more creative in our teaching. The energy that we used to expend trying to dream up learning activities in the absence of coordinated learning objectives can be reinvested in coming up with cool ideas about how to give students active work on the objectives we've crafted; or in giving useful feedback to students and thus enhancing the human qualities of the class; and more.

So although this may not look like improving online instruction, it's like the mostly invisible work that takes place when putting in the infrastructure of a new building. It's the proper first focus of any steps toward excellence.

In the next installation, we'll connect course materials and tech tools to the learning objectives and see how the edifice of objectives, activities, assessments, materials, and tools can create a coherent and therefore meaningful experience for everyone.