How much research has been done on flipped learning? Update for 2019

Every year here on the blog, I do a mini-lit review to take the temperature of flipped learning research. Specifically I go to the ERIC database and look for the number of peer-reviewed articles from the previous year that have "flipped learning", "flipped classroom", or "inverted classroom" in either the title or the abstract. The sum total of these three searches gives a number that works as a reasonable proxy for how much research is out there on flipped learning. I started this back in 2016 (to get the totals through the end of 2015), updated it in 2017, and updated again in 2018. Today, like Christmas in mid-year, I've updated this search again to have a look at how things went in 2018.

If you want to play along at home, just go to ERIC and do a query on

(title:"flipped learning" or abstract:"flipped learning") and (pubyear:2018)

Then redo it by replacing "flipped learning" with "flipped classroom" and then "inverted classroom".

In previous years, the data have come back with the same consistent message: Sustained exponential growth. However, last year I noted that while the exponential growth was still ongoing, the pace was slowing down. This year... well, folks, things are a little different.

Here are the results of the queries for 2018, along with comparisons to 2017:

If you want to access the raw data or cite this in a paper, use the DOI https://doi.org/10.6084/m9.figshare.5882818 (or click the link to go to the data on Figshare).

Here's the data in chart form:

It's a bit questionable to fit the data with an exponential model this time, but I did it anyway in Desmos:

The model here is $y = 0.346202(1.42929)^t$ and the $R^2$ value is 0.9235 for what it's worth.

Now, let's talk about these 2018 results.

So, 2018 was the first year since 2005 that we've seen an actual decline in research output on this subject. (And from 2004 to 2005, that "decline" was a drop from 1 paper to 0 papers.) It's only a decline of 12 papers compared with last year, but when you've been expecting 60% annual growth but then get a drop of any size, it's pretty noticeable. That reverse-hockey-stick in the first graph above is pretty brutal. Last year, I predicted using my models that there would be around 250 publications in 2018.  Ouch.

While it's disappointing to me personally that the predictions of past models fell way short of reality, I don't think there's any reason to believe these are leading indicators that flipped learning as a concept is in trouble or decline. Here are two good reasons why.

First, the overall trend of flipped learning research growth since 2000 is unambiguously exponential growth. To see this, here's a modified chart that shows the cumulative number of peer-reviewed articles on flipped learning over the same time frame as the first chart:

As you can see, despite the drop in 2018, the running total in orange is clearly the same sustained exponential growth we've come to expect. I ran the exponential regression on the cumulative total, and the model is $0.0876436(1.64482)^t$ with an $R^2$ of 0.986. So there is some reason to believe that 2018 is an outlier and that, overall, flipped learning research is continuing a history of strong growth.

Second, the data on publications for this year as of June 4 are showing a healthy pace, on par with 2017 and possibly 2018. I ran the query for publications in 2019 so far, through June 3, and the total number so far is 52. When I did this mid-year for 2016 that number was 38; for 2017 it was 50. Last year it was significantly higher – 79 – but I also waited until the end of July, so that number could be significantly higher than through June 2018. If I'd found a big drop in mid-year publications, for example the total through June 1 was around the same as for 2010, then that would be a real concern.

So I believe that flipped learning, and the research on it, is very much alive and kicking despite a break in the pattern of growth we've seen. I certainly see this anecdotally among my colleagues and other faculty elsewhere — a strong curiosity about moving to flipped learning and lots of questions from people who have tried it and want to improve their practice.

For what it's worth, here are three possible explanations for the dropoff in publications:

1. Flipped learning has reached the "plateau of productivity".  According to the Gartner hype cycle, new technologies progress through five stages, shown in this chart on the horizontal axis:

The peak for flipped learning probably happened around 2014, which is a couple of years after Jon Bergmann and Aaron Sams published their best-selling book on flipped learning and started getting massive media attention. (It's also right at the "elbow" of the exponential growth curve.) I think we started the "slope of enlightenment" about two years ago. This was when we started seeing systematic treatments of best practices and methodologies for flipped learning – my book was published in 2017 for example, and the Flipped Learning Global Initiative took off around the same time and offered professional certifications for educators on flipped learning.

It's possible that in 2018 we entered the "plateau of productivity" where flipped learning has been widely adopted (relatively speaking) and a lot of the original "frontier" of flipped learning has been explored, and so the number of research publications can be expected to decline – but not out of lack of interest or adoption. Far from it! Rather, you can think of the research publications graph as something like the absolute value of the first derivative of the hype cycle graph – the steeper the hype cycle graph, the faster the expectations are changing for practitioners and therefore the more research interest there is in exploring questions about its use. So if we plateau, due to settling in and using flipped learning on a regular basis with broad adoption, the slope is closer to 0.

2. Researchers on flipped learning are taking more time to do better-quality research. I've noted before that in flipped learning research as everywhere else, quantity doesn't equal quality, and in fact there are a lot of really bad flipped learning studies in print out there. One of the "grand challenges" for flipped learning that I outlined here was to create a body of research that is truly world-class – and this takes time. Perhaps, and I certainly hope this is the case, there are just as many people doing research on flipped learning, but taking their time to do it better than before – so the output may drop but the quality of the work will go way up.

3. The search query itself and the terminology being used may be in an adjustment process. I noticed that queries for "flipped learning" brought back more results in 2018 than they did in 2017, whereas "flipped classroom" and "inverted classroom" brought back less. This makes me wonder if the drop in output is due to a consolidation of terminology, toward – finally, after all this time – using a single term to describe what we mean. It also raises the possibility that earlier search queries were double-counting things. For example if a paper had both the terms "flipped classroom" and "flipped learning" in the abstract, this would show up as two distinct hits in the query. Earlier research on flipped learning might certainly do this; you can imagine an author writing something like Flipped learning, also known as "the flipped classroom", is a big idea these days... Whereas now as the concept becomes more mainstream, there are fewer such passages in use and so we find fewer hits in the search queries. I've always said that my methods in these update articles are just back-of-the-napkin calculations!

What I take away from this year's update is that flipped learning research is still going strong when you take the broad view, and there are signs that we're entering a new phase of practice with this concept. I'm not going to make any predictions for 2019 this time (we saw how that turned out last year) but I will say I am excited to see how this plays out.