What Happened Following Comprehensive Developmental Education Reform in the Sunshine State? The Impact of Florida’s Developmental Education Reform on Introductory College-Level Course Completion

Florida’s Senate Bill 1720 allowed many students to bypass developmental education and enroll directly in introductory college-level courses. We use an interrupted time series design to introductory college-level courses enrollment and passing rates in English and math for three cohorts of college students prereform and three cohorts postreform. Based on a cohort-by-cohort comparative analysis, we find that cohorts after the reform are more likely to enroll and pass introductory college-level courses in their 1st year of college, indicating that the reform may help to accelerate student success in college. Further, we find that Black and Hispanic students experience even greater gains in passing rates than White students, effectively narrowing the racial/ethnic achievement gap.

S tudents who arrive at college underprepared have historically faced an "obstacle course" of developmental education (DE) requirements upon college enrollment, which many never even begin or complete (Baily et al., 2010). Students are required to pay for these courses but do not receive any college credit. They also must complete these DE courses before enrolling in the gateway courses that fulfill degree requirements, thus slowing their academic momentum. In response to these challenges, many states and institutions have begun implementing a variety of different policies to reform their DE programs (e.g., Edgecombe, 2011). In this article, we look at policy impacts from the state of Florida, which implemented an extensive statewide reform that made DE optional for the majority of incoming students at the 28 state colleges in the Florida College System (FCS, the former community college system), and also required the colleges to change the instructional modalities for remaining DE courses to allow students to progress more quickly into college credit courses. Colleges were also required to develop a plan for providing enhanced advising and support services for students.
We begin by providing a brief overview of DE and recent reform measures across the country. Then, we detail the components of Florida's DE reform efforts and the theory of action underlying how these efforts may improve students' acceleration through introductory college-level courses (often termed gateway courses). Next, we describe our methods, which use an interrupted time series design to examine whether student success in introductory college-level courses during the 1st year of college has changed since the reform, and a comparative interrupted time series to study differential changes by race/ethnicity. After presenting our results, we offer a general conclusion regarding student success in introductory college-level courses following major DE reform in Florida.

Related Literature and Context
Historically, prior to enrolling in introductory college-level courses, many students across the country were first required to take DE courses in reading, writing, and/or math. Indeed, estimates suggest that over half of students enrolled in an associate degree program were required to take at least one developmental course (Bailey et al., 2010;Complete College America, 2012).
In Florida alone, historical estimates indicate that up to 70% of first-time-in-college (FTIC) community college students were enrolled in at least one DE course (Underhill, 2013). And additional studies have shown that Black and Hispanic students tend to be enrolled in developmental courses at higher rates than White students (Attewell et al., 2006;Perry et al., 2010;Ross et al., 2012). Unfortunately, very few students enrolled in developmental courses ever attempt, let alone pass, an introductory college-level course (Bailey et al., 2010;Complete College America, 2012). Thus, for many students, DE courses were often the main obstacle in achieving the academic momentum necessary to be successful in postsecondary education. Still, some studies have shown that DE may benefit some students, particularly those scoring the lowest on traditional college placement tests (Boatman & Long, 2018).
In order to better serve all students, many states have turned to reform measures that alter the placement and/or instruction of DE, with the goal of accelerating students into introductory college-level courses (Rutschow & Meyer, 2018). Reform measures in other states have focused on policies that accelerate students through DE and/or limit students' time in DE. Thus, the goal of DE reform is often to ensure student success in the introductory college-level courses, regardless of prior academic preparation. The theory of action underlying Florida's Senate Bill (SB) 1720 is that the reform efforts will increase student success in college by changing state policy directives, institutional programs and practices, and student decision-making.
Prior to Florida's DE reform, approximately 70% of FTIC students scored below college-ready on a statewide placement test and were required to take at least one DE course (Underhill, 2013). Under SB 1720, college students were exempt from college placement testing and developmental coursework if they entered a Florida public high school in 2003-2004 or later and went on to graduate with a standard high school diploma, or if they were active-duty members of the military. These students were assumed to be college-ready and could enroll directly into gateway courses in math and English. Prior research has shown that nearly one quarter of college students are misplaced into their first college course when placement decisions are based on test scores alone, most of these students being unnecessarily placed into DE courses (Scott-Clayton et al., 2014). Thus, making DE optional may reduce the financial and opportunity costs to students of being placed into courses they may not need, and may improve their academic progression.
The second component of the reform required colleges to change the way that DE courses are offered by implementing compressed, contextualized, modularized, or corequisite modalities. Compressed courses meet more frequently over fewer weeks, which potentially allows students to complete two sequential courses in the same semester. There is some evidence that compressed courses are associated with positive effects on short-term outcomes such as gateway course completion, as well as longerterm outcomes like credit accumulation and degree completion (Hodara & Jaggars, 2014). Contextualized courses present remedial material in an applied manner related to the student's intended major. Although there is limited rigorous research on this modality, there is some evidence that these courses may improve students' course performance by making the material more relevant to students' interests and career goals (Perin, 2011). Modularized courses assess mastery of course standards and then allow students to complete customized modules for only the standards in which they did not demonstrate mastery. This often results in students completing fewer credits in DE courses than they would otherwise need (Kalamkarian et al., 2015), and experimental evidence suggests that students are just as successful in modularized courses as more traditional DE models (Weiss & Headlam, 2019). Lastly, corequisite courses allow students to take developmental and college-level courses concurrently. This approach has been shown to increase the percentage of students completing college-level courses in the first semester or 1st year of college in other states (Denley, 2015;Jones, 2015;Kalamakrian et al., 2015).
We note that some of the colleges had been experimenting/ piloting the use of some of these instructional strategies in the years leading up to the reform (Park et al., 2016). As such, our identification strategy, which we discuss in the next section, accounts for any underlying upward trends that may be present in student outcomes prior to the reform that could be due to early implementation of this component of the reform.
The third component of the reform required colleges to provide enhanced advising services to increase student awareness of the developmental course options, as well as additional support services like tutoring for underprepared students. College students who receive enhanced advising tend to take and pass more courses (Scrivener & Weiss, 2009), and are also more likely to take advantage of tutoring and other types of academic support services (Visher et al., 2010). Support services typically tend to be underutilized by college students, even though they increase the likelihood that students will persist in college courses and tend to improve students' college GPAs (Perin, 2004).
Specific to Florida and from a broad, overall policy impact perspective, our earlier work has shown some initial promising results from Florida's DE reform. In the 1st year after SB 1720, the predicted probability of both taking and passing gateway courses increased by 5.3 percentage points in English and 3.7 percentage points in mathematics relative to the year prior to the reform. Further, our own work has shown that passing rates increase at a more rapid rate for Black and Hispanic students, compared to White students (Park et al., 2018). However, our work to date has been limited to one cohort of student outcomes following the reform. What is not yet known is whether these trends continue in the multiple years following the implementation of the reform. In other words, although the DE reform produced positive results for one cohort of students, did this continue to be the case for additional cohorts?
In this article, we employ a more rigorous approach and additional years by comparing three cohorts of students before and three cohorts of students after the reform. The balanced panel of 3 years of data prior to the reform and 3 years of data after the reform allows us to examine whether the reform has had an impact on student success for multiple cohorts of students following its implementation. Specifically, we examine enrollment rates as well as passing rates in two different ways: course-based passing rates and cohort-based passing rates. Whereas course-based passing rates are a more traditional notion of passing rates (the number of students passing the course expressed as a share of the number of students enrolled in the course), cohort-based passing rates (the number of students passing the course expressed as a share of the total number of students in a cohort of incoming students) provide a more global view of policy impacts. Specifically, cohortbased passing rates are a measure of the total impact of the policy and its ability to successfully increase student success. According to a recent study by Belfield et al. (2019), 1st-year measures of gateway course enrollment and completion rates are key predictors of longer-term student success and may be particularly important for explaining racial equity gaps in college completion rates. The following research questions guided this study: (1) How have enrollment rates, course-based passing rates, and cohort-based passing rates in introductory collegelevel courses changed following the DE reform among FTIC students enrolled in the FCS institutions? (2) How did the change differ for students of different racial/ ethnic backgrounds?

Data
We use data from Florida's K-20 Education Data Warehouse (EDW), which tracks all Florida public school students remaining in-state from kindergarten to postsecondary education. For these analyses, we use data from college enrollment records, student background characteristics, and indicators of high school preparation. We include data from all 28 FCS institutions in Florida. We include six cohorts of FTIC students, three cohorts who were enrolled prior to SB 1720 (entering cohorts Fall 2011-Fall 2013), and three cohorts who were enrolled after (entering cohorts Fall 2014-Fall 2016). For enrollment rates in college-level courses, we calculated the percentage of students enrolled in these courses, disaggregated by subject, as the share of the cohort of FTIC students who began their studies in FCS institutions in a given fall term. Then, for introductory college-level course passing rates, we used two different measures. The first is a course-based passing rate, calculated as the percentage of students passing the course as the share of the total number of students enrolled in the course, disaggregated by subject. The second measure is the cohort-based passing rate, which is calculated as the percentage of students passing the course as the share of the total number of students in the entire cohort. Thus, the cohort-based passing rate captures the overall effect of the policy and ascertains whether more students have been successful at taking and passing introductory college-level courses from the perspective of a cohort-by-cohort comparison. In addition, we also have measures of high school academic preparation, gender, race/ethnicity, and free/reduced-lunch eligibility. Descriptive statistics, by cohort, are presented in Table 1.

Analytic Approach
We use a time series design to examine whether there have been any changes in students' enrollment and completion of introductory college-level courses before and after the reform. We use a balanced panel with 3 years of data prior to the reform and 3 years after the reform. We assess the likelihood of enrolling in and passing college-level courses in the 1st year of college. In English, the first introductory college-level course is English Composition 1 (ENC1101). In math, we examine results separately for Intermediate Algebra (MAT 1033), which counts for elective credit only, and gateway math courses that fulfill the transfer associate's degree requirements in math (College Algebra, MAC1105; Liberal Arts Math 1, MGF1106; Liberal Arts Math 2, MGF1107; and Introductory Statistics, STA2023). For this first analysis, we use all of the students in each cohort as our full sample.
To empirically investigate the overall impact of the reform on enrollment and passing rates, we estimate the model below for student i at college j in year (cohort) t: In this specification, β 1 captures the change in the course enrollment/passing rate in the postreform period, β 2 is a vector of student background characteristics, β 3 is a vector of high school academic preparation indicators, ξ j is a college fixed effect to account for unobserved heterogeneity across institutions, and λ t is a continuous year (cohort) indicator to account for any underlying temporal trends. 1 By including the time trend, we are able to net out any impacts that may have resulted from changes other the 2014 implementation of the DE reform (such as early adoption of the new instructional modalities by some of the colleges) that could have impacted student outcomes (Jacob et al., 2017). In other words, by including a continuous year indicator, β 1 will only capture an increase in student outcomes that can be attributed directly to the policy and not to any prepolicy trend that could have continued after the policy was implemented. However, one limitation of this approach is that there could be other unobserved changes in the postpolicy years that may have affected the outcomes, even in the absence of the policy. Further, in order to explore whether there were differential changes in enrollment/passing rates by race/ethnicity, in an additional set of analyses, we include indicators for Hispanic and Black students (White students form the comparison group; students of other race/ethnicity are excluded from these models) and interact these indicators with the postreform indicator.
We estimate the following model for student i at college j in year (cohort) t: Under this specification, β 1 captures the change in course enrollment/passing rate in the postreform period; β 2 and β 3 capture the overall difference in outcomes for Black and Hispanic students, respectively; β 4 and β 5 capture any differential changes in student outcomes for Black and Hispanic students, respectively, postreform; β 6 is a vector of student background characteristics; β 7 is a vector of high school academic preparation indicators; ξ j is a college fixed effect to account for unobserved heterogeneity across institutions; and λ t is a continuous year (cohort) indicator to account for any underlying temporal trends. This model captures the base changes in overall student outcomes between the prereform period and the postreform period, as well as whether there has been a differential change for Hispanic and Black students. In this second analysis, we only include White, Black, and Hispanic students. Before presenting our findings, we note that although it is tempting (as well as potentially quite interesting) to investigate the impacts of individual components of Florida's DE reform, we also see perhaps greater merit in examining the impacts of the policy as a whole. We also note that previous attempts at isolating impacts of components of DE reform have concluded that is difficult to disentangle these effects when states implement DE reform policies that contain several elements, such as those in North Carolina, Virginia, and Florida (Bickerstaff et al., 2016). Thus, we remind the reader that our results should be interpreted as overall policy effects for incoming cohorts of FTIC students in the FCS, including both exempt and nonexempt students. In addition, we note that the overall effect of the reform may be greater than the sum of its parts, further warranting a holistic analysis. Put differently, there may be a form of synergy that emerges from the three major components of the reform that adds to the impact in a way that is greater than the individual impacts of the three components added together. More to that point, our most recent findings from an annual survey to campus leaders across the FCS found that the DE reform brought about institutional transformation that extended far beyond the requirements of SB 1720 (Mokher, Spencer, et al., 2020). These findings suggest that Florida's DE reform brought about comprehensive reform that has created an increased culture of student success. As such, we are interested in whether the reform had impacts for all incoming FTIC students as well as whether these impacts varied by race/ethnicity.

Descriptive Portrait
We begin with descriptive portrait highlighting enrollment and passing rates for introductory college-level courses. We present findings for all students, disaggregated by subject. Additional findings, disaggregated by race, can be found as an online supplement to this article at centerforpostsecondarysuccess.org. Table 2, 1st-year introductory college-level course enrollment rates increased following the implementation of the DE reform. Immediately after implementation, introductory college-course enrollment rates increased by 6.49, 8.69, and 3.20 percentage points for English composition, intermediate algebra, and our pooled measure of gateway math courses, respectively. We note that we are modeling intermediate algebra and gateway math courses as different outcome measures due to the fact that some students, depending on their intended major, may be required to take intermediate algebra (a collegecredit bearing course) as a prerequisite course prior to enrolling in a gateway math courses that will satisfy the math requirement for the associate's degree. Further, this increase continued into the 2015 cohort before experiencing a slight plateau in the 2016 cohort. Course-based passing rates. As shown in Table 3, the 1st-year course-based passing rate for English composition remained relatively constant over time, although the number of students entering the course increased dramatically following the implementation of SB 1720. The course-based passing rate for intermediate algebra decreased by nearly 5 percentage points in 2014 but rebounded somewhat in the 2015 and 2016 cohorts. Just as with English, though, the number of students enrolling in intermediate algebra increased dramatically in the postimplementation cohorts. Course-based passing rates in gateway math courses declined slightly following the implementation of SB 1720.

Enrollment rates. As shown in
Cohort-based passing rates. As shown in Table 4, the number of students both taking and passing introductory college-level English and math courses in their 1st year increased following the reform. Cohort-based passing rates in both English composition and intermediate algebra saw immediate high jumps in the 1st year postreform, increasing by 4.36 and 3.44 percentage points, respectively. College-level math courses had a moderate passing rate increase of 1.77 percentage points in the 1st year postreform, but they experienced a higher increase in the 2nd year postreform, with a jump by 5.37 percentage points from 2013. All courses reached a plateau in the 3rd year postreform.

Regression-Adjusted Results
We move now to our regression-adjusted results under our interrupted time series/comparative interrupted time series designs. For ease of interpretation, we report our findings as predicted probabilities and marginal effects, calculated from the regression results. Full regression results are available from the authors.
Enrollment rates. All of the increases in the 1st-year introductory college-level course enrollment rates following the implementation of the DE reform were statistically significant. English enrollment rates increased by 4.79 percentage points, while   (Table 5). Further, all of the differential effects by race/ethnicity were also statistically significant and positive (Table 6). Not only did all students enroll in introductory college-level courses at higher rates following the implementation of the reform, but Hispanic and Black students increased their enrollment at faster rates than White students, narrowing the enrollment gap for Black and White students and effectively closing the enrollment gap for Hispanic and White students. In English composition, for instance, while White students experienced an increase of 7.47 percent points in their enrollment rate, Hispanic and Black students experienced 12.75 and 22.64 percentage point gains, respectively. A similar pattern also occurred in intermediate algebra and gateway math courses.
Course-based passing rates. Table 7 presents the results from our set of models predicting overall 1st-year course-based passing rates in introductory college-level courses before and after the implementation of Florida's DE reform. Overall, course-based passing rates in English composition (ENC1101) and gateway math course passing rates (MAC1105, MGF1106, MGF1107, and STA2023) remained the same following the reform. In intermediate algebra, however, course-based passing rates declined by nearly 5 percentage points. Table 8 presents the results from our set of models predicting differential course-based passing rates by race/ethnicity. In English composition, White and Hispanic students experienced a slight increase in course-based passing rates (1.25 and 0.54 percentage points, respectively), and Black students experienced a decline (0.67 percentage points). Further, in intermediate algebra, whereas all students experienced a decline, both Hispanic and White students had similar (non-statistically significant from each other) declines of around 4 percentage points, and Black students had declines of 6.07 percentage points, roughly 2 percentage points more than White and Hispanic students. Finally, White and Hispanic students experienced no significant change in course-based gateway math courses, and Black students experienced 4.3 percentage point decline in course-based passing rates. Table 9 presents regression-adjusted 1st-year cohort-based passing rates before and after the implementation of Florida's DE reform. Overall, we find that all three indicators for cohort-based passing rates increased significantly following the reform. Specifically, cohort based-basing rates increased by 3.38, 3.48, and 2.94 percentage points in English composition, intermediate  algebra, and combined gateway math courses, respectively. Of note is that following the reform, over half of the incoming cohorts were successfully taking and passing gateway English courses within the 1st year, with nearly one in five students taking and passing gateway math courses. Table 10 presents the results from our set of models predicting differential cohort-based passing rates by race/ethnicity. Across the three panels, students of all racial/ethnic backgrounds experienced increased cohort-based passing rates in their English and math courses, with Black and Hispanic students seeing even  greater increases than White students. For instance, Black students' passing rates in English composition increased by 14.18 percentage points, which is 7.89 percentage points higher than the increase for White students. Further, Black students experienced gains that were 3.27 percentage points higher than White students in intermediate algebra. We note that in terms of gateway math, the base gain for Black students (4.79 percentage points) appears smaller than that for White students (4.97 Note. Significance reported for differences prereform/postreform. *p < .05. **p < .01. ***p < .001. percentage points); however, we note that the marginal effect gauging the difference between these changes is not statistically significant. Put differently, White and Black students appear to have experienced similar prereform to postreform gains in terms of 1st-year, cohort-based passing rates for gateway math.

Discussion and Conclusion
Florida's DE reform sought to provide learning opportunities more tailored to students' needs by doing away with traditional placement models and making DE optional for many students while revamping the way DE courses are taught and enhancing advising and student support services to help students make good choices and continue on a path for success. The result is that many more incoming students are taking and passing introductory college-level courses in their 1st year of study, and preexisting achievement gaps in these courses for Black and Hispanic students narrowed. We argue that these outcomes are a result of the three components of the reform working in tandem and that, although not all students were directly affected by each component, all three are essential to achieve increased student momentum.
Making DE optional (the first component of the reform) resulted in a significant enrollment effect-many more students enrolled in introductory college-level English and math courses when DE was no longer required. Two groups of students were now directly enrolling in college-level courses: those who would have been able to do so previously based on the traditional placement model in effect and those who would have previously been required to first take DE courses. Thus, not only did enrollments increase, but introductory college-level classrooms also become increasingly heterogeneous in terms of student academic ability, since students could enroll directly in these courses regardless of prior academic preparation. Although there could have been unintended negative consequences given the curricular and pedagogical challenges associated with educating a more academically diverse group of students, we instead observed an overall increase in the total number of incoming students taking and passing introductory college-level courses once the reform was implemented. This suggests that at least some of the students who would have previously been required to take DE under the previous placement model were indeed successful in passing the collegelevel course without first taking DE. This is not entirely surprising given that previous studies have demonstrated that traditional placement models often falsely assign students to DE courses when they would have been capable of passing the college-level course without first taking DE (Leeds & Mokher, 2020;Scott-Clayton et al., 2014). Thus, one sensible policy lever available to states seeking to increase student momentum is to do away with traditional placement models and make DE optional. However, Florida's policy did more than this, and we stress caution in adopting any DE reform that fails to address the needs of a third group of students: those who may still benefit from DE courses.
Florida's reform also contained two other major components: new instructional strategies for DE courses and enhanced advising and student support services. Importantly, we note that Florida's comprehensive DE reform was intended to support students at all levels of preparation-those on the margins of college readiness who could directly enroll in gateway courses, as well as those further behind who could still take DE courses designed in new ways to help them progress more quickly to college-level courses. Thus, the new instructional strategies for DE courses are an important mechanism for increasing academic momentum for those students still required to take DE and those who elect to take the courses even though they are optional. Guiding much of this process is the other important mechanism by which students received benefits from the reform: enhanced advising and student support services. The reform also put additional advising supports into place to help ensure that students made good choices. We argue that adding these two additional components (new instructional strategies for DE courses and enhanced advising) made for a more sensible reform choice than just eliminating DE completely, which could have harmed the least-prepared students.
Although we argue that Florida's reform functions as an interconnected set of three reform measures, and although we note that it may not be entirely possible to disentangle the impacts of specific components, we see a need for additional research that probes deeper into the components of the reform and how different students are impacted by the reform. For instance, although we found that gains in college-level passing rates were the greatest among nonexempt students who received the new DE instructional strategies and the enhanced support services (Mokher, Park-Gaghan, et al., 2020), we have not yet identified which instructional strategies are the most effective. And although we found that students who had the lowest levels of high school academic preparation tend to have the greatest gains in college-level course passing rates (Park-Gaghan et al., 2019), we have not yet studied the long-term educational trajectories of students who take, but do not pass, college-level courses. Future research is certainly warranted on Florida's DE reform, particularly in looking at more distal outcomes, such as transfer and degree completion, particularly given that prior research has indicated that many DE reform efforts have had limited impacts on longer-term student outcomes (Jaggars & Bickerstaff, 2018). Finally, although not a direct focus of the legislation, another positive externality that warrants further investigation is the closing of the preexisting racial/ethnic achievement gap in introductory college-level course completion. Future research is certainly warranted that examines equity in other academic outcomes, particularly those more distal.
Taken collectively, the findings from this investigation suggest that DE reform can have a real and significant effect on increasing the academic momentum in introductory collegelevel courses, particularly for traditionally underrepresented groups. Although we do not yet know the long-term impacts of the reform, we are encouraged to see that the share of students completing introductory-level math and English courses continues to remain at all-time highs, even 3 years after the initial implementation of the policy. We encourage other states and contexts to consider the DE reform measures implemented in Florida A visual inspection of the data suggested that certain student outcomes were on a slight upward linear trajectory prior to the implementation of the policy. As robustness tests, we also included squared, cubic, and exponential trend variables; the selection of the trend variable did not affect the results in any meaningful way.