Randomized controlled trials of U‐Pace instruction: Outcomes in two gateway courses
Abstract
Less than optimal student success in gateway courses figures prominently in college noncompletion. Past findings suggest that U‐Pace instruction holds promise for increasing student success. However, all published studies of U‐Pace instruction were conducted in the gateway course context of introductory psychology. The objective of this research was to rigorously evaluate the efficacy of U‐Pace instruction in introductory sociology and introductory political science for students at‐risk for college noncompletion and students not at‐risk. The findings from both randomized controlled trials indicate that regardless of risk status, U‐Pace students outperformed their conventionally taught face‐to‐face counterparts earning a greater percentage of final grades of A or B and higher scores on a proctored cumulative assessment of learning, independent of final grades. The results not only confirm previous findings of greater academic success and greater learning associated with U‐Pace instruction but also clearly demonstrate that U‐Pace instruction produces greater learning and greater academic success. Further, the findings of this research reveal that these student outcomes generalize across courses in two disciplines for both students at‐risk for college noncompletion and students not at‐risk, providing strong support for the efficacy of U‐Pace instruction.
Lay Description
What is already known about this topic
- U‐Pace online instruction combines mastery learning and proactive instructor support.
- Greater academic success and greater learning have been associated with U‐Pace instruction.
- Previous studies evaluated U‐Pace instruction in the context of introductory psychology.
What this paper adds
- This is the first study to examine U‐Pace instruction in other gateway course contexts.
- U‐Pace produced greater academic success and learning than conventional, Face‐to‐Face instruction in sociology and political science.
Implications for practice and/or policy
- The findings strongly support the efficacy of U‐Pace regardless of undergraduate risk status.
- Could strategic implementation of U‐Pace in core courses facilitate degree completion?
1 INTRODUCTION
Suboptimal success in gateway courses (i.e., earning a final grade of C, D, or F) is a pattern well known to administrators, advisors, and instructors in higher education, and substantial attention and resources are often directed towards increasing success in this context. This is because successful performance in gateway courses is critical to students' capacity to perform well in advanced coursework, which builds upon the foundational knowledge acquired in gateway courses. Thus, success in gateway courses is central to students' college completion and has implications for the social and economic strength of U.S. communities, which has been shown to be associated with the proportion of citizens who hold a college degree (Baum, Ma, & Payea, 2013).
Students at‐risk for less than optimal success in gateway courses and as expected at‐risk for college noncompletion include students disadvantaged by low family income, factors associated with racial/ethnic minority status, or academic underpreparedness. Less than half of students with family incomes below $25,000 who enrol in a 4‐year college attain the baccalaureate within 6 years whereas over 75% of students with family incomes of $100,000 or more graduate within 6 years (U.S. Department of Education, National Center for Education Statistics, 2011). Similarly, the 6‐year graduation rates are lower for African American (40.9%), Hispanic (53.5%), American Indian/Alaska Native (41%), and Pacific Islander (50%) students than White students (63.2%; U.S. Department of Education, National Center for Education Statistics, 2015). Further, students who are academically underprepared, as demonstrated by low college admission test scores, may be more likely to attend an institution that accepts a large percentage of applicants. Institutions with lower levels of admissions test (ACT/SAT) selectivity have lower graduation rates than institutions that are more selective (ACT Inc., 2015).
Effective instructional interventions that can increase success in gateway courses for both students at‐risk for college noncompletion and students not at‐risk are of paramount importance to increasing the number of students who attain a college degree. U‐Pace instruction has shown promise in increasing success for both students at‐risk for college noncompletion and students not at‐risk (albeit in a single gateway course).
1.1 U‐Pace instruction
U‐Pace is an instructional intervention that allows students to control the pace of their own learning, combining mastery learning and proactive instructor support in an online learning environment. Mastery learning has been shown to improve student academic outcomes (Block & Burns, 1976; Kulik, Kulik, & Bangert‐Drowns, 1990; Rae & Samuels, 2011; Svenningsen & Pear, 2011). The course content in U‐Pace instruction is divided into small units which students are expected to master over the course of a semester. As with many mastery courses, students work through the units one at a time and cannot move on to new material until they have demonstrated mastery (Block & Burns, 1976; Rae & Samuels, 2011; Svenningsen & Pear, 2011), defined in U‐Pace instruction as scoring a 90% or higher on the associated unit quiz. As with other mastery‐based courses, students build onto their knowledge as they progressively master each unit while working at their own pace. This mastery requirement provides students the opportunity to fully work through the concepts they have not yet mastered and to practice their understanding of concepts through repeated testing. This approach is supported by previous research, which has shown that repeated testing improves retention of material, including complex concepts (Karpicke & Roediger III, 2008; McDaniel, Howard, & Einstein, 2009; Roediger III & Karpicke, 2006).
As students work to master the course concepts, they are proactively supported by the instructor with weekly assistance provided to all students without them having to ask for it. Instructors provide students with individually tailored e‐mail messages containing instructional support (concept feedback, study skills, course information) and elements aimed at modifying students' beliefs about themselves as learners, with the goal of shaping student behaviour towards success. Instructor expectations and student beliefs about the malleability of intelligence—that their ability to master difficult content can grow through hard work—have been found to impact student achievement (Mueller & Dweck, 1998; Rosenthal, 1994; Rosenthal & Jacobson, 1968; Wilson & Linville, 1982, 1985; Yeager & Dweck, 2012). In U‐Pace instruction, instructors express unwavering confidence in their students' ability to master the material even if students are struggling to advance to new course material. Student beliefs about the causes of their difficulties in mastering difficult content also influence achievement; students who endorse understanding that their difficulty is temporary have greater achievement than students who believe their difficulty is permanent (Wilson & Linville, 1982, 1985). In U‐Pace instruction, instructors set high expectations for students (90% mastery requirement) and communicate unwavering belief that their students can meet these expectations and overcome difficulty through hard work and persistent efforts. Thus, proactive instructor support promotes positive student beliefs about their learning and positive course outcomes.
As part of the proactive instructor support, U‐Pace instructors provide individualized feedback on concepts, which students have previously found challenging, and on study skills, students can utilize to be more effective in the course. Research has found that students benefit from interaction with their instructors (e.g., Cox, McIntosh, Terenzini, Reason, & Quaye, 2010; Komarraju, Musulink, & Bhattacharya, 2010; Sax, Bryant, & Harper, 2005) and from individualized conceptual feedback about course material during the learning process (e.g., Gallien & Oomen‐Early, 2008). Individualized feedback has also been found to be particularly effective in changing behaviour (Noar, Benac, & Harris, 2007). Proactive instructor support is conceptually similar to proactive (sometimes called intrusive) advising (Varney, 2007, 2012), in that it provides instructors a strategic method to reach students and shape their behaviour and allows instructors to directly provide individualized expectations and information for students from the very beginning of their interactions. Instructors can change the rate of feedback if students need greater support or are working through the self‐paced material quickly. Instructors use student performance and engagement information standard to learning management systems used throughout higher education and freely available U‐Pace templates for supporting students to guide the creation of personalized and effective proactive support messages. U‐Pace instruction helps students take control over their learning so that they develop a deep understanding of course concepts and achieve greater academic success.
Previous research has shown U‐Pace instruction to be successful when implemented with students in introduction to psychology courses. A significantly greater percentage of U‐Pace students was found to earn final grades of A or B compared with students in the conventionally taught face‐to‐face sections of the course (Fleming et al., 2016; Reddy et al., 2013). Further, compared with conventionally taught introduction to psychology students, U‐Pace students also scored significantly higher on cumulative exams measuring core course concepts, suggesting better retention of course material for students in U‐Pace instruction (Fleming et al., 2016; Reddy et al., 2011). Introduction to psychology was chosen as the course context for the initial studies of U‐Pace instruction because this gateway course enrols a large number of first‐year students and students from a variety of academic disciplines (Adelman, 2004). Past research has only examined U‐Pace instruction in the context of introduction to psychology. Whether the findings of greater learning and greater academic success would hold in other course/disciplinary contexts has never been investigated.
1.2 The current research
Two randomized controlled trials (RCTs) were conducted to rigorously evaluate the efficacy of U‐Pace instruction in the contexts of introductory sociology and introductory political science. Although previous research has not examined U‐Pace instruction's efficacy in a gateway course other than introductory psychology (Fleming et al., 2016; Reddy et al., 2013), U‐Pace instruction's empirical grounding supports the assertion that U‐Pace instruction would produce greater academic success for students across course/discipline. Individualized mastery learning courses have been used to improve student outcomes in diverse subject areas (e.g., Block & Burns, 1976; Kulik, Kulik, & Bangert‐Drowns, 1990; Rae & Samuels, 2011; Svenningsen & Pear, 2011), and proactive instructor support could be applied in a variety of academic disciplines and educational settings.
The current RCTs were also undertaken to determine whether U‐Pace instruction produces greater learning and greater academic success for both students at‐risk for college noncompletion and students not at‐risk because this is a key question with clear implications for higher education. Students at‐risk for college noncompletion in the United States include academically underprepared students (reflected in low scores on college admission tests), economically disadvantaged students (Pell grant eligible), and students disadvantaged by factors associated with racial/ethnic minority status (Snyder, de Brey, & Dillow, 2016).
Specifically, both RCTs investigated whether the greater academic success and learning associated with U‐Pace instruction in introduction to psychology would generalize to the introductory courses in sociology and political science. It was hypothesized that students randomly assigned to the U‐Pace sections in each disciplinary course, regardless of risk status, would be more academically successful, earning a greater percentage of final grades of A or B, and perform better on a proctored, cumulative exam (which served as an independent assessment of learning that did not count towards their final course grade) compared with students in the comparison sections of these courses taught by the same instructors, covering the same content and using the same textbook within each discipline.
2 METHOD
An RCT for each gateway course was conducted at a public doctoral‐granting university with approximately 24,000 undergraduates. The university admits up to 75% of applicants. The RCTs were approved by the Institutional Review Board for the Protection of Human Participants. Informed consent was obtained for participation in the RCT and separately for participation in a proctored cumulative exam assessing learning independent of course requirements. Exam scores did not contribute to final course grades.
2.1 Recruitment and participants
All undergraduates interested in taking the gateway course were invited to participate in the corresponding RCT through e‐mail announcements. Participants agreed to be randomly assigned to either a face‐to‐face or online section of the course. They were informed that they would receive $150 (approximately equivalent to the cost of a new textbook) even if they dropped or withdrew from the course and that normal policies and dates for dropping or withdrawing from the course and tuition would apply. Exclusion criteria included under age 18 or repeating the course. Both were verified with institutional records.
For each RCT, students were randomly assigned (using a stratified procedure to evenly distribute students at‐risk for college noncompletion across instructional conditions) to yield a potential sample of 480 students: 2 instructors × 2 instructional conditions (U‐Pace instruction vs. Face‐to‐Face instruction) × 2 semesters × 60 students per section. Students from low income backgrounds (Pell grant eligible), racial/ethnic minority backgrounds showing gaps in graduation at most U.S. institutions (African American, Hispanic, Native American, or Southeast Asian), having low standardized test scores for college admission (ACT composite scores of less than 19), and/or cumulative college grade point averages less than 2.0 on a 0.0–4.0 scale were defined as at‐risk for college noncompletion.
2.1.1 Sociology RCT
Altogether 1,103 undergraduates took introductory sociology. This enrolment includes all sections of introductory sociology, both in and outside the sociology RCT. A total of 460 undergraduates consented to participate in the sociology RCT. The actual sample size was 440. A total of 16 students could not be randomly assigned due to: holds on student enrolment due to tuition nonpayment (n = 2), academic holds (n = 1), verified to have been repeating the course (n = 2), course schedule conflicts precluding assignment to Face‐to‐Face instruction (n = 9), and addition of the three credit course in the study would create a credit overload (n = 2). Student characteristics by instructional condition and in the overall sociology RCT sample are presented in Table 1.
| U‐Pace | Face‐to‐Face | Total | |
|---|---|---|---|
| N | 218 | 222 | 440 |
| % at‐risk | 56.2 | 54.7 | 66.3 |
| Mean age | 20.77 | 20.39 | 20.57 |
| (SD) | (3.97) | (3.93) | (3.95) |
| % female | 73.0 | 66.2 | 69.5 |
| % freshmen/sophomores | 71.4 | 75.0 | 73.2 |
| % first gen. students | 59.9 | 63.0 | 61.5 |
- Note. Due to missing data, sample sizes for comparisons vary slightly across variables. RCT: randomized controlled trial.
2.1.2 Political science RCT
A total of 800 undergraduates took Introduction to American Government and Politics (the introductory political science course). Of this number, 503 consented to participate in the political science RCT. The actual sample size was 444. Given miscellaneous issues, 59 students could not be randomly assigned: holds on student enrolment due to tuition nonpayment (n = 4), academic holds (n = 7), verified to have been repeating the course (n = 4), or under age 18 (n = 2), course schedule conflicts precluding face‐to‐face assignment (n = 34), addition of the three credit course in the study would create a credit overload (n = 8). Table 2 presents the characteristics of the students in the political science RCT by instructional condition and in the overall sample.
| U‐Pace | Face‐to‐Face | Total | |
|---|---|---|---|
| N | 219 | 225 | 444 |
| % at‐risk | 76.4 | 67.6 | 69.0 |
| Mean age | 22.95 | 21.25 | 20.57 |
| (SD) | (6.41) | (5.53) | (3.95) |
| % female | 68.0 | 60.5 | 64.2 |
| % freshmen/sophomores | 64.5 | 44.9 | 54.8** |
| % first gen. students | 62.1 | 68.5 | 65.4 |
- Note. Due to missing data, sample sizes for comparisons vary slightly across variables. RCT: randomized controlled trial
- ** p < 0.01.
2.1.3 U‐Pace instructional condition
In the U‐Pace instructional condition in both sociology and political science, content was delivered in 12 small units, with two associated quizzes each, totalling 24 quizzes. Students were required to demonstrate mastery (score at least a 90%) on each unit quiz before advancing to the next unit. Each quiz contained 10 multiple‐choice questions to be completed in 7 min. Students had unlimited attempts to complete the quizzes; however, they had to wait at least 1 hr between quiz attempts and the quiz questions changed with each attempt. Students were also provided with a score for their quiz but were not informed which questions they answered incorrectly, as the goal was for students to master concepts rather than learn answers to specific quiz questions. Quiz questions were developed to assess understanding beyond rote memorization.
Students received weekly proactive instructor support delivered through e‐mail messages, which instructors tailored for their students with the assistance of field‐tested e‐mail templates. These instructor‐initiated support messages provided concept feedback and included elements to shape student behaviour towards success and modify student beliefs about their ability to learn. Instructors personalized the messages by reviewing student quiz scores and course engagement information recorded in the learning management system. Instructors also offered office hours for students as is typical with most university courses.
2.1.4 Face‐to‐face instructional condition
In both the sociology RCT and the political science RCT, students in the Face‐to‐Face instructional condition attended two on‐campus 75‐min classes weekly, which combined lecture, discussion, and interactive activities. Students were not required to demonstrate mastery of course material. Students did not receive proactive weekly support e‐mails from the instructors. However, instructors provided help with concepts and assistance as they normally would (i.e., they interacted with students during class, by e‐mail, by appointment, during office hours, and exam reviews).
2.1.5 Instructors, training, and fidelity
Instructors were experienced faculty in the disciplines in which they were teaching the introductory courses. In each RCT, there were two instructors who each taught both the U‐Pace and Face‐to‐Face instructional conditions. Course content and textbook were held constant within each RCT. All instructors were trained to implement the instructional conditions with fidelity. Condition implementation was examined throughout the study by holding biweekly conversations with the instructors and monitoring the learning management system course sites and e‐mails. Instructors were also provided with an implementation manual, which included field‐tested proactive instructor support e‐mail templates. Instructors recorded on an ongoing basis the time spent on instructional activities (e.g., preparing for class, e‐mailing with students, and helping students in person) using a log, adapted from Worley and Tesdell (2009). The logs indicated that the instructors invested an equivalent amount of time in each instructional condition. High fidelity to instructional approach in both RCTs was found when the classes and course sites were checked. The instructors saved every e‐mail message sent to and received from students in separate e‐folders for each instructional condition. Bi‐weekly examination by the research team revealed high fidelity to instructional approach. For example, no students in the Face‐to‐Face instructional condition received proactive weekly support messages, whereas all students in the U‐Pace instructional condition did. The proactive support messages sent to students in the U‐Pace instructional condition were assessed to be high quality communications that aligned with the templates specifying that the e‐mail messages should contain help with concepts not yet mastered, behaviour‐based praise to shape student performance towards success, and elements to modify students' beliefs that they could be successful even if unsuccessful at the moment. Further, bi‐weekly conversations with the instructors throughout the RCTs demonstrated fidelity to instructional condition.
2.2 Measures
2.2.1 Academic success
In both RCTs, academic success was assessed using the institutional record of final course grades. Students' academic success was defined as a final grade of A or B because a 3.0/4.0 GPA is often considered the cut‐off for selective programmes of study and is the minimum needed for receiving academic honours. Further, a final grade of B is the threshold for a high likelihood of graduating (e.g., Denley, 2016). Final grades were objectively determined based on student performance on the mastery quizzes (U‐Pace instructional condition) or exams (Face‐to‐Face instructional condition) drawn from the same pool of assessment items. Final course grades for students randomly assigned to the Face‐to‐Face instructional condition were based on their mean score on four multiple‐choice exams given across the semester. Grades for students randomly assigned to the U‐Pace instructional condition were determined by the number of quizzes a student completed with at least a 90% score.
2.2.2 Learning
In both RCTs, separate from performance in the course (not contributing to the final course grade), learning was measured by a proctored cumulative exam developed by subject matter experts. For ethical reasons, all students were invited to take the cumulative exam at the end of the course and receive the $25 incentive. Half of the students from both the U‐Pace instructional condition and the Face‐to‐Face instructional condition were randomly selected to take the cumulative exam so their performance could be compared with the students who were not selected a priori and also took the exam. Students taking the exam were encouraged to do their best and were required to work on the multiple‐choice exam for at least 30 min, write their name on the exam, and check their answers before leaving.
2.3 Statistical analyses
Analyses were conducted separately for each course RCT (sociology and political science). Analysis of variance was used to examine differences in performance by instructional condition and risk status on the proctored, cumulative assessment of learning, independent of final grades. Performance on the proctored exam for students who were randomly selected a priori to take the exam and those who were not randomly selected and took the exam was also compared using analysis of variance. Students' final grades by instructional condition and risk status were compared using binary logistic regression. The percentage of students earning final grades of A or B (indicating academic success) was compared with the percentage of students earning final grades below B, including students who withdrew from the course in the group with final grades below B. The percentage of students who withdrew from the course by instructional condition was compared using chi‐square analysis. Chi‐square analysis was also performed to examine whether the instructional conditions differed in the percentage of at‐risk students after students withdrew from the course sections.
3 RESULTS
3.1 Sociology RCT
The percentage of students who withdrew from the course did not differ by instructional condition, χ2 = 0.066, p = 0.797: U‐Pace attrition = 8.56%, 19/222 students; Face‐to‐Face attrition = 7.80%; 17/218 students. Total attrition was 8.18%, 36/440. After students withdrew from the course, the percentage of at‐risk students did not differ by instructional condition, χ2 = 0.148, p = 0.700: U‐Pace students at‐risk = 56.2%, 114/203; Face‐to‐Face students at‐risk = 54.7%; 110/201.
3.1.1 Academic success
Objectively determined final grades of A or B versus B through F were compared for at‐risk students and students not at‐risk in the U‐Pace and Face‐to‐Face instructional conditions using binary logistic regression (see Figure 1). Instructional condition predicted academic success. The odds of earning a final grade of A or B were 5.43 times higher for U‐Pace students than Face‐to‐Face students, Wald χ2 = 38.231, p < 0.001, Exp(B) = 5.43. At‐risk students were not significantly less likely to earn a final grade of A or B, Wald χ2 = 2.966, p = 0.085, Exp(B) = 0.587, than students not at‐risk. No interaction was found between instructional condition and risk status, Wald χ2 = 0.228, p = 0.633, Exp(B) = 0.803.
3.1.2 Learning
U‐Pace students (M = 74.65) scored significantly higher on the proctored, cumulative exam, the measure of learning independent of final course grades, than Face‐to‐Face students, M = 66.12, F(1, 329) = 31.18, p < 0.001, η2 = 0.087. At‐risk students (M = 62.21) scored significantly lower on the proctored, cumulative exam than students not at‐risk, M = 68.56, F(1, 329) = 5.716, p = 0.017, η2 = 0.017. No interaction between instructional condition and risk status was found, F(1, 329) = 0.004, p = 0.948, η2 < 0.001. Figure 2 depicts these findings. When including only the students who were randomly selected a priori to take the cumulative exam in the analysis, the same finding held. That is, the U‐Pace students scored significantly higher than the Face‐to‐Face students. The mean exam scores were U‐Pace = 75.29 and Face‐to‐Face = 61.37, F(1, 68) = 13.523, p < 0.001, η2 = 0.166.
3.2 Political science RCT
As in the sociology RCT, no difference in the percent of students who withdrew from the course was found between the U‐Pace and Face‐to‐Face instructional conditions, χ2 = 2.395, p = 0.122: U‐Pace attrition = 9.13%, 20/219 students; Face‐to‐Face attrition = 5.33%; 12/225 students. Total attrition was 7.21%, 32/444. After attrition, the percentage of at‐risk students did not differ by instructional condition, χ2 = .641, p = 0.423: U‐Pace students at risk = 69.4%, 152/219; Face‐to‐Face students at‐risk = 64.0%; 144/225.
3.2.1 Academic success
Final grades of A or B versus B through F were compared for at‐risk students and students not at‐risk in the U‐Pace and Face‐to‐Face instructional conditions using binary logistic regression (see Figure 3). Both instructional condition and risk status predicted academic success. The odds of earning a final grade of A or B were 14.29 times higher for U‐Pace students than Face‐to‐Face students, Wald χ2 = 66.136, p < 0.001, Exp(B) = 14.29. At‐risk students were significantly less likely to earn a final grade of A or B, Wald χ2 = 6.932, p = 0.008, Exp(B) = 0.405, than students not at‐risk. No interaction was found between instructional condition and risk status, Wald χ2 = 0.883, p = 0.766, Exp(B) = 0.855.
3.2.2 Learning
As in the sociology RCT, U‐Pace students (M = 73.90) scored significantly higher on the proctored, cumulative exam than Face‐to‐Face students, M = 66.90, F(1, 306) = 25.624, p < 0.001, η2 = 0.0774. Additionally, at‐risk students (M = 68.53) scored significantly lower than students not at‐risk, M = 72.51, F(1, 306) = 8.523, p = 0.004, η2 = 0.027. No interaction between instructional condition and risk status was found, F(1, 329) = 0.464, p = 0.496, η2 < 0.002. The findings are shown in Figure 4.
When including only the students who were randomly selected a priori to take the cumulative exam in the analysis, the same finding held. That is, the U‐Pace students (M = 76.86) scored significantly higher than the Face‐to‐Face students, M = 69.19, F(1, 78) = 7.389, p = 0.008, η2 = 0.087.
4 DISCUSSION
The findings are from two RCTs in the course contexts of introductory sociology and introductory political science. As such, this is the first study to rigorously evaluate the efficacy of U‐Pace instruction in course contexts other than introductory psychology. Students at‐risk for college noncompletion constituted more than half of the RCT samples. Consequently, the efficacy of U‐Pace instruction for at‐risk students and students not at‐risk could be determined. The results not only confirm previous findings of greater academic success and greater learning associated with U‐Pace instruction (Fleming et al., 2016; Reddy et al., 2011; 2013) but also clearly demonstrate that U‐Pace instruction produces greater learning and greater academic success than conventional face‐to‐face instruction. Further, the findings of this research reveal that these student outcomes generalize across courses in two disciplines for both students at‐risk for college noncompletion and students not at‐risk, providing strong support for the efficacy of U‐Pace instruction. Given that the context of higher education in the United States is different from many parts of the world, the findings of this RCT may not be generalizable to many nations.
It is particularly noteworthy that U‐Pace instruction produced greater learning and greater academic success in introductory sociology and introductory political science as undergraduates must acquire the fundamental skills and knowledge in such foundational high enrolment courses to have a high likelihood of degree attainment. If students fail to acquire the essential learning outcomes in gateway courses, they will be less prepared to learn concepts in advanced coursework that rely on these earlier concepts and may not perform well. The findings from both RCTs indicate that regardless of risk status, U‐Pace students outperformed their conventionally taught counterparts earning both a greater percentage of final grades of A or B and higher scores on a proctored cumulative assessment of learning, independent of final grades. Success in gateway courses potentiates undergraduates' ability to perform well in subsequent coursework and complete college. That U‐Pace instruction produced striking outcomes that were consistent in two foundational courses for both at‐risk students and students not at‐risk may have policy implications: (a) exposure to U‐Pace instruction in multiple courses may amplify benefits (i.e., academic success and learning) and, perhaps, students' understanding that they are capable of college‐level work, and (b) the success achieved in multiple U‐Pace courses may accelerate college credit accumulation and reduce time to degree and total cost, especially if offered in the first 60 college credits, a critical period for student success and prevention of nonretention. A limitation of the current study is that the effects of multiple U‐Pace course experiences on student outcomes was not studied. This would be a fruitful direction for future research.
Enrolling in no less than 15 credits each semester, multiterm course plans, early identification of an intended major or a meta‐major (affinity cluster of majors), and proactive advising, where students are armed with information based on predictive analytics to gauge the likelihood of successful completion of degree pathways, have all been advocated and are currently being implemented in postsecondary institutions in the United States as strategies to increase college completion (Education Advisory Board, 2015). Could strategic implementation of U‐Pace courses and the resulting success as suggested by this research also facilitate college completion? In general, the online learning environment provides access to students returning to college or enrolling for the first time, as these students often have time constraints due to family obligations such as supporting dependents and working full‐time. Beyond access, the high performance expectations in U‐Pace instruction coupled with proactive instructor support, occurring in a self‐paced online learning environment, may also produce success for such undergraduates. Although U‐Pace instruction was found in this research to produce greater learning and academic success for both students at‐risk for college noncompletion and students not at‐risk, the effect of U‐Pace instruction on student perceptions is important to understand. Previous research (Fleming et al., 2018) showed that exposure to U‐Pace instruction increased students' sense of control over learning and perception of instructor support from the beginning of the semester to the end. Investigating the efficacy of U‐Pace instruction for students enrolled in community and technical college would be useful directions for future research. Students enrolled in open‐access institutions typically must overcome greater challenges (e.g., weaker academic preparation) to be successful than students enrolled in 4‐year institutions. The setting for this research was a broad‐access university accepting 75% of applicants, which suggests that the findings may hold for community college students, although this is an empirical question.
Although this research has notable strengths, including randomization of more than 400 students to each RCT, an examination of outcomes in two disciplines and by risk status, the combined RCTs do not provide enough statistical power to uncover the mediator(s) underlying the effects of U‐Pace instruction. Perhaps U‐Pace instruction works by strengthening students' sense of control over academic challenges as suggested by Fleming et al. (2018). As perception of control improves students' academic engagement may increase, possibly paving the way for greater learning and greater academic success. Further investigations identifying the mediator(s) would be important as it could lead to other ways to improve outcomes in educational environments, both formal and informal. Future studies could also compare U‐Pace instruction with emerging models of online instruction such as adaptive learning models, which are compatible with the personalization and self‐pacing of U‐Pace instruction.
In summary, this research extends previous findings on U‐Pace instruction by providing experimental evidence that U‐Pace instruction produces greater learning and greater academic success than Face‐to‐Face instruction across gateway courses in two disciplines, importantly, for students at‐risk for college noncompletion and students not at‐risk. Less than optimal student success in gateway courses figures prominently in college noncompletion and the findings of this research suggest that U‐Pace instruction holds promise for increasing student success.
ACKNOWLEDGEMENTS
We thank the instructors for their participation in the project and offices throughout the institution that supported our ability to conduct the randomized controlled trials. The work was supported by the Institute of Education Sciences (IES), U.S. Department of Education, through Grant R305A110112 and by the Fund for the Improvement of Postsecondary Education (FIPSE), U.S. Department of Education, through Grant P116Q140006. The opinions expressed are those of the authors and do not represent views of IES, FIPSE, or the U.S. Department of Education.
