When trying to inform college students about campus resources they might explore to help boost their grades, limited email reminders work better than other methods, a recent study from Oregon State University found.
However, those email “nudges” are only the first step, researchers say.
“It’s worthwhile to nudge students toward academic resources, but don’t stop there,” said lead author Todd Pugatch, associate director of OSU’s Department of Economics in the College of Liberal Arts. “That will only get you so far in a certain segment of the student population, and these low-cost nudges can’t be a substitute for making a more personal relationship with students and getting a sense of the challenges they’re facing.”
For the study, published in the Journal of Human Resources, Pugatch and co-author Nicholas Wilson from Reed College tested different forms of advertising on 2,119 students enrolled in introductory Economic Principles classes at OSU during the 2018-19 school year.
They notified students of three academic support services: tutoring, coaching and extra practice problems. Half of the students in the experimental group received these nudges via email and half via text message. Additionally, half of the students were entered into a lottery to receive a $250 credit at the campus dining halls and bookstore if they accessed the support services before a set date.
Researchers also varied the frequency and timing of their messaging, spreading them out across weeks three, six and nine of the ten-week academic term. About 43 percent of students in the experimental group received two messages, and 13 percent received three.
Researchers chose the introductory economics class because it has been highlighted as a problematic undergraduate course. It fulfills a requirement for 40 majors at OSU but has a high rate of “DWF” (a grade of D or F or withdrawing from the course), so it hinders students’ ability to graduate on time.
The study’s findings were modest but significant: Students were more likely to click on links sent via email than via text message, and the most frequently used resource were the extra practice problems offered online by the course instructor.
Emails increased service awareness by nearly 20 percent, and the optimal number of emails was two. Sending a third email late in the term actually backfired in some cases, Pugatch said. Neither text messages nor the added financial lottery incentive made any dent in student use of support services.
The results also highlighted a consistent struggle around capturing college students’ attention. Two-thirds of students in the study were classed as “always attentive,” 30 percent “never attentive” and 5 percent or fewer as “compliers.” Only the compliers appeared to shift their behavior in any way based on the email/text nudges, meaning that the “never attentive” students were still unreached by the study’s efforts.
“Of the students who got at least one of these messages, either email or text, nearly one-third of them at the end of class reported ‘I’ve never heard of these services before.’ So that’s really discouraging,” Pugatch said. “Going beyond these electronic forms of communication will be necessary to reach a large subset of the student population that just isn’t responsive.”
Researchers surmised that the reason students were more likely to engage with extra practice problems than other forms of support was the ease of access: The practice problems were available online and at any time of day, whereas tutoring and coaching services had to be accessed on campus during specific hours.
The study was conducted in the 2018-19 school year, but Pugatch said the COVID-19 pandemic would probably render this type of email reminder less effective, partly due to the increase in overall electronic communication students are receiving and partly because of mental health challenges facing students.
Overall, email reminders are a relatively quick, low-cost way to encourage students to seek academic support, but they should be seen as just one tool in an instructor’s toolbox, he said. More personalized messages, sent to individual students by name and triggered when they are in trouble academically, have been found to be more effective in other research on this topic.
