Research report: How does hyflex work in a large course?

The hyflex concept seems to be picking up momentum as we further explore the uncharted water that is Fall 2020. For the uninitiated, "hyflex" is a portmanteau of "hybrid flexible" and refers to a course design included parallel face-to-face (F2F) and online structures, and students are allowed to engage with the course by choosing between those structures, or choose both structures, on an ongoing basis.

I'm reading as much as I can about hyflex courses right now. In an earlier post, I reported on a research article that studied a hyflex course situated in a small course taken by Social Work majors and minors. But what about large general education courses? Can hyflex work there too, and what are the issues involved? That's the focus of this article:

Sowell, K., Saichaie, K., Bergman, J., & Applegate, E. (2019). High Enrollment and HyFlex: The Case for an Alternative Course Model. Journal on Excellence in College Teaching, 30(2), 5-28.

More information and a direct link to the article here, although  you need a subscription to access it.

What is this study about?

As the article title suggests, this study focuses on a hyflex implementation of a course with a high enrollment: A general-education Nutrition class taught at the University of California - Davis, enrolling 550 students per academic quarter and 2000 students per academic year as of 2014. Ordinarily, the in-person meetings were Tuesday/Thursday for 80 minutes each and primarily involved lectures with embedded low-stakes quizzing. There were also some outside projects like a food diary project and an extra-credit "Jeopardy"-like quiz game.

The size of the class played a central role in its redesign: Because enrollment was high and increasing at the time, UC-Davis was running out of space. It sets the UC-Davis course apart from the social policy course that we considered in the Malczyk study last time, where there were only 18 students involved, all of whom were specialized Social Work majors and minors. The primary pedagogy in the other course, too, was discussion rather than lecture, and the primary driver for hyflex was to provide flexibility in scheduling.

What did the researcher(s) do?

To convert the UC-Davis course to hyflex, the designers essentially just digitized the main elements of the F2F meetings and put those online to create a parallel online course. The in-person lectures were recorded to video and, interestingly, also to audio as podcasts. The extra credit game show was adapted to an online format. But most other elements of the course -- the diet project, in-person office hours, etc. -- were left unchanged, and all students still needed to be in class to take the two midterm exams and the final exam.

Normally when speaking of hyflex courses, there are online and F2F versions of the course, and students are allowed to choose at regular intervals which version they are going to use. But in this case, students didn't have this choice: They were allowed to sample the online and F2F versions of the course during the first week to determine their preference, but then they had to choose either in-person or online attendance to class activities for the remainder of the quarter. After this selection, you were either a "F2F student" or an "online [OL] student", and no switching was allowed.

Further, access to the materials and class sessions was restricted. Access to in-person lectures during week 1 was on a first-come/first-served basis (remember the space limitations), and students who declared themselves to be "online" students were discouraged from attending in-person classes. Conversely, F2F students were locked out of the video-recorded lectures through the LMS, and even the online students could only view the video lectures for seven calendar days following the date the lecture was given.  (However, all students -- both F2F and online -- had unlimited access to the audio files from the lectures throughout the semester.)

According to the authors, the reason for the restriction on classroom attendance was the lack of space. On the other hand, the restrictions on access to online materials  was never fully explained in the paper other than to say that granting online students only 7-day access to video recordings of lectures was "to encourage students to keep up with course material".

Finally, the course had six teaching assistants who, together with the instructor, offered 12 office hours a week open to all students. Additionally, the discussion board tool Piazza was implemented to give students an online forum for questions and discussion.

Having redesigned this Nutrition class in hyflex format, the course ran for the 10-week Winter quarter in 2014. Apart from the design itself, the authors made two interventions during the quarter:

• They gave a survey to students in weeks 1 and 2 asking for demographic information, prior online course experience, and their preferred formats for learning; and
• They gave a survey at the end of the course with 19 questions on their lecture attendance, use of materials, experience with their chosen course format, and their overall format preference.

What did the researcher(s) find?

Of the 691 students enrolled, 592 students consented to the study; 209 of those chose the online format and 383 chose the F2F format. Close to 3/4 of the students in either modality (online or F2F) had never taken a course with video-recorded lectures before. Over 60% of the online students were third- or fourth-year students; the level of the F2F students was more or less uniformly distributed.

Here are some of the standout results from the surveys:

• The main reasons given by the online students for choosing that modality were a preference to work from home (46% said this was why they chose online) and to resolve schedule conflicts (25.3%).
• When asked at the beginning of the course about their preferred learning format, the top results were attending in-person lectures (selected as the top choice by 59% of students), viewing recorded lectures (only 11.4%), reading the textbook (9.4%), and one-on-one discussions with TAs and instructors (8.5%).
• If you group those numbers by the modality that students would eventually choose, there are some interesting patterns. The top three preferred learning formats for students who would eventually choose F2F interaction were attending in-person lectures (at 69.3%); reading the textbook (10%); and discussions with TAs and instructors (7.2%). Only 3.8% of the eventually-F2F student said watching video recorded lectures was their preferred learning method. For the eventually-online students, the top three were in-person lectures, but only 38.9% said this; video recorded lectures at a relatively huge 26.5%; and discussions with TAs and the instructor (10.5%).
• At the end of the course, the students were asked the same question. The F2F students still mostly preferred in-person lectures (70.4% this time), followed by reading the textbook (11.2%). But the third-place preference shifted from discussions with TAs or the instructor -- the percentage dropped from 7.2% to just 0.5%. The new third-place preference was, ironically, viewing video-recorded lectures, which went from 3.8% preferred before to 6.8% after.^[1] The first and second choices for the online students switched places, with video-recorded lectures being preferred by 44.4% of online students and in-person lectures by 32.9%. And just like the F2F students, the third-place choice was no longer discussions with TAs (dropping from 7.2% to just 1.4%) and instructors; the new third-place preference was reading the textbook (10.1%).
• The online students weren't prohibited from attending in-person lectures, only discouraged from doing so. The data showed that 68% of the online students only watched the video-recorded lectures, so 32% of those students attended at least one in-person lecture. In fact, the study found that of the online students who attended at least one in-person lecture, 86.6% attended between 15 and 18 in-person lectures. There were only 18 of these in-person lectures in all! Many of those students (38) attended the in-person lecture first, mainly either because they needed a second coat of lecturing to complete their notes or because they were not native English speakers.

Here's a summary of the before-and-after results, remixed somewhat from how it appears in the paper. The bolded data represent the top 3 preference in the pre- and post-course surveys.

What do these results mean for others?

These results tell a fascinating story about student preferences. Every student enters a course with a belief about how they learn best. What we see in this study is that students largely chose the modality that most closely resembles those beliefs. But as the course unfolded, those beliefs played out in some non-intuitive ways:

• The most striking change to me is the profound shift away from "one-on-one discussions with the TA and professor". The preferences for this mode of learning simply fell off of a cliff. About 1 out of 10 students believed that this was their preferred way of learning entering the course; by the end of the course it was closer to 1 out of 100. And the biggest proportional drop was among the F2F students, who you would think might prefer F2F courses at least partially for the personal attention that F2F instruction affords.
• In fact the F2F students preferences increased for learning methods that seem distinctly removed from close personal interactions: reading a textbook and watching recorded video.
• While the F2F students' primary preference stayed largely the same, the online students' top preference flipped from in-person to recorded videos --- and the preference for recorded video at the end of the course was not far from double that at the beginning.

What I'm seeing in the results is a cautionary tale. We should be respectful of students' learning preferences, but also aware that these preferences are often a cocktail of biases, misinformation, and fixed-mindset limiting beliefs that can change in unexpected ways when they come into contact with actual experience with different learning modes. For example, many of the students in this study said that they preferred 1-1 interactions with their TAs and professors and largely did not prefer to learn via recorded lectures; but in the end, with 10 weeks of experience in a real learning environment, 1-1 interaction was almost at the bottom of the list, and the preferences for online video nearly doubled.

So it seems wise to take a "trust but verify" approach to student preferences. Listen to them but remember that preferences change with experience, and preferences shouldn't be a pretext for avoiding experiences, especially those that get students out of their comfort zones.

In particular --- and the following is not a popular opinion --- we need to beware of preferences held by us or our students for in-person instruction. I fear that many decisions about opening campuses this fall to face-to-face teaching will be driven by a preference for being in a room with other people, or felt beliefs about online teaching, rather than scientific data about the effectiveness of online learning or the spread of disease. Preferences are all well and good, but they are not the same as data, and just because we feel we learn or teach best in one particular way doesn't mean that it can't be done just as well or better in another.

Limitations and questions

For a "hyflex" design, this course has a distinct lack of "flex" to it. Students couldn't switch modalities during the semester, which seems to be the distinguishing characteristic and main selling point of hyflex. So there are some concerns about external validity here and whether the preferences really reflect authentic experiences. On the other hand, the course was set up this way because UC-Davis had space shortages and too many students for the spaces they had. This seems familiar these days, and we head into Fall 2020 semester under social distancing and other pandemic-related constraints, that's probably closer to home than we think.

I'm not thrilled with the restrictions on access to course materials from an instructional standpoint, and I think those restrictions had an impact on the results. It seems hard to imagine how students could have an informed opinion about the value of online video if access to them is restricted or even blocked. Giving only a 7-day window for online students to access the recorded videos of the lectures in order to "encourage students to keep up with course material" seems patronizing and unhelpful (especially since all students had unlimited access to the audio of those lectures). It also doesn't work, as those students can easily save those videos and use them as long as they want, or distribute them to the F2F students to route around the LMS lockout.

Finally, keep in mind that like the Malczyk study, this study does not look at any differences student learning between the online and F2F groups, or within those groups between the start and end of the semester, or between this class and a corresponding non-hyflex section of the course. This would be a good jumping-off point for a follow-up study.

How is it possible that the F2F students could have increased preference for watching recorded video, when they were locked out of the recorded videos on the LMS? It could be several things. Those students might be friends with online students who let the F2F students watch videos through the online students' accounts. The F2F students might be watching videos other than the ones made for the class. The F2F students could simply be jealous of the online students' access to recorded video and adopting a belief that if only they had access too, they would learn better. There's no way to know without more data, but one thing is clear: Restricting access to video doesn't prevent students from watching video, or wishing they could. ↩︎