How much research has been done on flipped learning? Annual update for 2018

It's now a tradition here at my blog to do an annual update of my answer to the common question, How much research is out there about flipped learning? I first posted about this two years ago after my book was published, and updated it last June to include info on 2016 and make some predictions about 2017. I've gone through and done it again this year, and I'd like to share the results of publication on flipped learning in 2017 and make some more predictions.

First, a reminder of the process. What I did in 2016 was go to the ERIC database, and start with the following search:

(title:"flipped classroom" or abstract:"flipped classroom") and (pubyear:2000)

This finds all articles published in 2000 with "flipped classroom" in either the title or the abstract. I then checked a box at ERIC that filters out everything but the peer-reviewed articles. Then I just made a note of how many such articles there were. I repeated this search using two other common handles for the flipped classroom: flipped learning and inverted classroom. So these three searches provide me one row of a spreadsheet (for the year 2000). I then redid all three searches 15 more times, for the years 2001 through 2015. To the master spreadsheet, I added a column that totalled up all the publications for a given year over for each of the three terms. The result graph appears in the original post. In 2017, I added another row by repeating the threefold search for 2016, and that chart appears in the update.

The first time I did this, I was blown away by the massive exponential growth in research publication on flipped learning starting around 2012. In the update last year, that exponential growth continued and I even fit an exponential regression model that indicated that the number of flipped learning publications was increasing at over 80% per year.

As I said, back in late June/early July I re-ran the search again to pick up the publications from all of 2017. Here's how it turned out. First of all, the numbers for 2017:

Here is the updated chart with the 2017 data added into the historical data from 2000---2016:

I re-ran an exponential regression model to the data like last year. The model (with $t$ scaled so that $t=0$ is the year 2000) is

$$N = (0.0319656281487) \cdot (1.6724629929)^t$$

The $R^2$ on this model is $0.9817$ which indicates a high level of statistical significance. Here are the data with the model superimposed:

I've uploaded the raw data to figshare so you can play with it yourself, and there's a DOI and a citation generator at the figshare page in case you want to use this in your own publications.

Before I unpack the results, let me reiterate some past caveats about my methods. This is not a truly rigorous literature search but more of an academic back-of-the-napkin approach. There could be a lot of double-counted articles in my search (e.g. if an article used both "flipped learning" and "flipped classroom" in its abstract). I also know for certain that this search method misses relevant papers that don't include any of the three search terms in the title or the abstract; I know because I was just reading a paper yesterday from 2015 on "flipped linear algebra" that didn't use any of the three search terms until the body of the paper. I also did what you should never do in a real literature search, which is use only one database. A rigorous search would be looking at half a dozen databases, more nuanced search terms, and filtering out results more carefully. So, take all this with a grain, maybe an entire box of salt.

That being said, here are some things I am seeing in the results along with some context about the papers in the search.

First, the exponential growth continues but at a slightly slower pace. Last year, looking at the 2000---2016 data the annual growth rate was a whopping 88.3%. After 2017 and updating the model, it's "only" 67.2%. I don't have any good explanations for that, and maybe this isn't an accurate enough modeling process for the difference to matter. I do know that I certainly wouldn't mind it if my retirement account were growing at 67.2% per year.

Doing the math on this, a growth factor of 1.6724629929 means that at this rate, the number of publications on flipped learning is doubling every 1.34 years, or roughly every 16 months. And you can really see this in the data --- at the end of 2015 the count was 89 which is less than half the count for 2017.

Second, this growth definitely appears to be continuing in 2018. I re-ran my threefold search described above with pubyear:2018 and the results for "flipped classroom", "flipped learning" and "inverted classroom" just for the first seven months of 2018 were 58, 21, and 0 respectively for a total of 79. When I did this mid-year for 2016 that number was 38; for 2017 it was 50. I did it a little later this year than in years past, but that's still a substantial increase in the mid-year numbers.

Another thing to notice, related to the above, is that it's really quite amazing how much flipped learning research has been done recently. Remember that at the ends of 2014, 2015, and 2016 the publication counts were 35, 89, and 131 respectively. Taking the mid-2018 numbers into account, which bring the publication grand total up to 271, this means that over half the research that has ever been published on flipped learning has been published in the last 18 months (i.e. in 2017 and 2018); over two-thirds of it in the last two and a half years; and almost 90% of it in the last three and a half years.

This is pretty mind-blowing; and you need to keep it in mind when people talk about "how little research has been done" on flipped learning. For example, many flipped learning papers start off by stating that very little research has been done on flipped learning (or flipped learning restricted to their particular context); this may not be true when you read the finished paper, but it definitely might have been true for the authors when they first submitted the manuscript 12-16 months ago. (Remember the doubling time.) Or, when skeptics talk about the paucity of research supporting flipped learning, they might be thinking of some op-ed or tweet they saw 2-3 years ago when the body of literature was one-quarter the size that it is now.

Finally just a couple of remarks about the papers themselves. I can't keep up with all of this research because it's coming in too fast; I have a Google Scholar alert set up for new papers and skim it daily, then pick off any that look particularly interesting for deeper reading. Based on my experience reading this stuff:

• As I mentioned in last year's update, the quality of the research being done is pretty uneven. I see papers as a reader that make me wonder how they made it past the review process; I see even more as an editor and reviewer that don't make it past the process. This is true for research in almost any field, so I don't worry about flipped learning research; in fact the homegrown nature of flipped learning research is one of its strengths. Still, though, keep in mind that a higher output of research means a higher output of bad research.
• The focus of this research seems to be shifting a little, in what I think is a really positive way. We seem to be moving past papers investigating whether flipped learning is better than traditional lecture, and more towards questions about what kinds of flipped learning environments work best for students --- how long videos should be, whether we should use video at all, how to optimize flipped learning for adult learners, and so on. Also there are more papers --- still a small number, but growing --- adding to the theoretical base of the subject, things like scoping reviews and papers about theoretical frameworks that set future research up on a firm foundation.

Now it's prediction time. Last year I took the mid-year numbers and projected forward with a guess, and I was close to exactly right. I ran some numbers, and this year for the first time I'm uncertain what's going to happen. If you just plug in $t=18$ to the exponential model, you get about 335 publications for 2018. That seems insanely unrealistic. But if you look at last year's numbers --- we were at 50 publications in midyear and ended with 131, which is 2.62 times the midyear count --- and apply the same growth rate to this year's midyear count, you get $2.62 \times 79 \approx 207$. That's a really wide variation, much moreso than in previous years. I'd guess that the final count will be somewhere in between, let's say 250. But I also think that next year at this time we'll see a big correction in the regression model. Stay tuned!