Reading research articles the GTD way

Confession time: I have always struggled with reading research artices. Dating all the way back to my early undergrad days as a psychology major, I've found it terribly hard to maintain focus and attention on journal articles. One time as a psych major, my professor sent me home with an article to read. I started reading it around 4:00 in the afternoon, fell asleep before I finished the abstract, and woke up at 1:00am. In graduate school for mathematics, in my job-related research on teaching and learning, in research for my book in 2015-2016, and especially now at my sabbatical at Steelcase, the story is the same: When I start reading research articles, I get bored, distracted, and sleepy, and I find it incredibly hard to make it through a paper --- even a short one --- and extract meaningful information from it.

Recently here at Steelcase, I started a project to do a comprehensive literature review of research on active learning classrooms. This is a big project, expected to take about 1/3 to 1/2 of the time that I spend at Steelcase for the next 3-4 months. And guess what this involves? Lots and lots of reading of research articles. The queue of papers to read currently sits at around 50, and I've been chipping away at it for about two weeks. But it's been tough sledding, just like it was in college (although I do manage not to take epic naps most of the time). And yet, unlike in college, this is my literal job, so I have to get it done and do it well. That means I have to figure out how to hack my brain and work around my struggles.

Projects, not tasks

A couple of weeks ago I started seriously thinking about how to overcome these issues. Pretty soon, I came to a conclusion that seems obvious in retrospect:

I have been treating reading a research article like a task, when in fact it's a project.

This is the language of Getting Things Done. In GTD, a task is a single atomic action. Projects by contrast are bundles of related tasks that all lead to a single conclusion. "Plan summer vacation" for example is not a single thing (i.e., a task) but a bundle of related things, completion of all of which will mean your vacation is planned. A basic rule of GTD is to examine each item that comes into your inbox and determine whether it is a task or a project. Confusing the two can lead to situations where you have "tasks" in your to-do list that are so big that they are too daunting or demoralizing to get done, so they just sit there.

I have written before about the importance of distinguishing projects from tasks in academia, especially as it pertains to grading. Grading a test from a class of 40 people is a project, not a task, and when we treat it like a task, it makes it almost impossible psychologically to engage with it. What I realized here, is that reading research articles is remarkably similar to grading in some ways. When I've conceived of reading an article as one task then it's so much work and so time- and brain-consuming that I found it hard to tackle. The natural coping mechanism is avoidance. My body and brain get tired and bored; or I look for excuses not to read; or I "read" superficially and check off the box from my to-do list having understood little from the paper.

How to read a research article

So as with grading, the key for me was to isolate what I see as the appropriate discrete steps to take. This article by anthropologist Jennifer Raff was a breakthrough for me. Using her suggestions as a starting point and modifying it to meet my own personal style, I have condensed the research-reading process down to eight steps.

Read the introduction --- not the abstract --- and identify the big guiding questions of the article. I'd always started reading an article by reading the abstract, but Raff points out that this isn't always a good idea. That's because the abstract always includes a summary of the results and they are always, magically, in harmony with the research questions, so reading the abstract may cause confirmation bias in you while you're reading. Instead, you use the abstract to determine whether the paper is of interest to you in the first place when you are putting it in your queue; and you return to it at the end. So skip the abstract and just read the introduction. While doing this, identify the big questions: Not the research questions, but the big questions that the entire field is trying to understand. At which of those big, wicked questions is this paper aiming?
Read and summarize the background and the literature review. That is, summarize the framework in which the study situates itself and especially try to understand the assumptions and potential biases that the study is making. If you're like me and doing a literature review yourself, or just getting started in the field, this is a good time to jot down any previously-published work that the current paper draws upon.
Summarize the specific research questions the paper addresses, and state the null hypotheses if it has any. Here, summarize the specific questions that the study is investigating. These are related to, but much more concrete than the questions from step 1.
Summarize the overall approach of the study and the methods used. Here we're looking for a high-level summary of how the paper approached investigating the questions, and not anything in depth. Raff suggests drawing diagrams to make this visual. She also points out that here, you do not need to understand the methods --- including the statistical analyses --- to the extent that you could replicate the study yourself. Just make sure you could explain the overall approach to another person. I love the point she makes, that the details of the methods and analyses are really more there for the reviewers than they are for the readers. How liberating is that?
Summarize the results that were found. Don't try to understand what those results mean yet --- just write down what they are.
Summarize the conclusions drawn by the researchers from the results. Again, not what you conclude from the results but rather what the researchers conclude. Those two things might be very different.
Loop back to the beginning and read the abstract. Now we go back and read the abstract, mainly to answer the questions: Did the researchers actually answer the research questions that they posed? And do the results actually support the conclusions they draw? The abstract will always make it look like the answers are "yes" and "yes", but as a reader you have to judge for yourself and use critical thinking.
Write out any evaluative notes that I have about the whole paper. These are free-for-all notes. Sometimes I write nothing; sometimes I write about connections to other papers I read; on one that I read yesterday I simply wrote "This paper is a mess". Anything I want to remember the paper by that has not already been written.

There's a ninth step that I sometimes do as well: Queue up all the papers in the references that I want to look up next. Especially since I am doing a literature review, I want to make sure I discover any papers further upstream from the one I'm reading and add those to my reading queue.

The benefits

I only wrote out the steps above a couple of weeks ago, so I'm still tweaking them. But I can already see three big benefits from breaking down my reading this way.

I don't have to think as much. These eight steps provide a scaffolding on the reading process that gives direction while reading --- simple areas of focus that are easy to keep in view, making it harder to drift off. They also make it more likely that reading research will fit my time available, energy level, and context. For example, if it's 2:45pm, I'm at about a 70% on the energy scale, and I have 15 minutes avaialble, I wouldn't have the time or enough fuel in the tank to read an entire article, but I could certainly pick one off the queue and read the introduction, or summarize the lit review of a paper whose introduction I had previously read, etc. You're simply more likely to do things that are simple and lightweight than those that are big and scary.
It gives me the ability to stop and restart later. On that note, these steps allow me to stop reading before the paper is completely done and then pick up the process later. You could do that without these steps of course, but these give a coherent framework with lots of designated "exits".
It makes it easy to take notes and write blog posts. I take notes at each of these steps in Evernote (more on how I use Evernote, Trello, and ToDoist for this in a later post) and those are just organized by the eight steps above. And once I've made detailed notes on each of the steps, I've found it's easy to condense them into blog posts like the one from last week. I wrote that post straight off my research notes and it was pretty effortless, because all the background research had been done and organized. (Here's the actual Evernote note I made while reading that particular paper.)

I've also found that by taking this GTD-flavored approach has allowed me to focus more on my reading and get a lot more out of it, which is really the end goal. Why didn't I think of this 25 years ago?

As always, I'd love to hear what you think of this, so please leave a comment if you have one.