In a new study, Batten’s Ben Castleman and two other UVA researchers investigated how the shift to online learning during COVID-19 has affected student success.

More and more students have been opting to take courses online over the past decade. But with the arrival of COVID-19 in March of this year, tens of millions found themselves thrown into the experience with little warning—and in many cases, for the very first time.

“I think it's without controversy to say that the abrupt shift to online learning brought about one of the most massive disruptions in the history of American education,” said Batten professor Ben Castleman during the latest edition of Batten Expert Chats.

Castleman, who also directs UVA’s Nudge4 Solutions Lab, spoke alongside Gaby Lohner and Kelli Bird from the School of Education and Human Development. The three researchers shared the results of their new study, which investigated how the sudden shift to online learning has affected students’ academic performance. Specifically, the study focused on the rate of course completion for Virginia community college students during the spring 2020 semester.

While the University of Virginia itself might seem like a natural source for such data, UVA and other similarly elite universities only enroll about 3% of students nation-wide. Community colleges, on the other hand, enroll approximately 40%—which means they better represent the country as a whole. It also means they can offer more information about disadvantaged student groups. “Community colleges disproportionately enroll lower income and first-generation students, who we know have borne the brunt of the economic, health, and childcare challenges due to COVID,” Castleman said. “We’re particularly interested in students who are in community colleges and navigating these other challenges.”

In order to study the effect of the shift to online learning, however, the researchers needed to isolate that shift from those other COVID-related obstacles. To do so, they looked at the performance of students within a given course, some of whom started the spring 2020 semester in-person and others who started online, and then compared changes in these students’ academic outcomes relative to in-person and online students from prior semesters.

Overall, the researchers found that the shift to online learning triggered a roughly 8% decrease in average course completion rates. Surprisingly, even when students took classes with instructors who had taught online before the pandemic, the decrease in academic performance remained relatively consistent. By contrast, rates of course completion varied significantly according to students’ academic standing: Students whose GPAs placed them in the lowest third of their class showed a 15% decrease in course completion, for example, while for those in the highest third that rate was only 3%.

“It seems that the struggle with the move to online was more on the student side, rather than based on instructor experience,” said Lohner.

Castleman expressed particular concern about the “equity implications” of the team’s research. Students from lower-income backgrounds are more likely to be struggling with issues that might interrupt or complicated online learning, such as unreliable Wifi, the loss of a job, or even a case of COVID-19 in their family. The pandemic could very well be exacerbating long-standing inequalities in higher education—which highlights the need for action in the policy realm, in Castleman’s view. The study’s findings “just increase the onus of responsibility on us as faculty members, as institutions, and as governments at the local, state, and federal level to double down on the investments we make to support and strengthen opportunities for students who are struggling in the face of COVID,” he argued.

For colleges and universities specifically, the way forward in our current climate is far from easy: They must weigh many factors when considering whether to reopen for in-person classes—the cost of making classrooms safe, the potential loss of tuition dollars, and the possible impact on public health, to name a few. “Our hope,” Castleman said, “is that our results inform what we recognize are very complex and difficult decisions.”