Assessing the First Two Years’ Effectiveness of Statway®: A Multilevel Model With Propensity Score Matching
Abstract
Objective: Statway is a community college pathways initiative developed by the Carnegie Foundation for the Advancement of Teaching designed to accelerate students’ progress through their developmental math sequence to acquiring college math credit in statistics. Statway is a multifaceted change initiative designed to address the complex problems that impede student success. Specifically, it is a one-year pathway program through which students acquire college math credit. Instructors use research-based learning principles to improve the content and pedagogy for student learning and incorporate social-psychological interventions to sustain student engagement and persistence. In addition, language supports for students’ accessibility to mathematics learning are integrated into the curriculum. Professional development resources assist faculty as they teach new content utilizing unfamiliar pedagogies. Statway is organized as a networked improvement community intending to accelerate educators’ efforts to continuously improve. This study was aimed to assess the effectiveness of Statway during its first two years of implementation. Method: We applied a multilevel model with propensity score matching to control for possible selection bias and increase the validity of causal inference. Results: We found large effects of Statway on students attaining college math credit with persisting effects into the following year as Statway students also accumulated more college-level credits. These improved outcomes emerged for each gender and race/ethnicity groups and for students with different math placement levels. Conclusion: This study provided robust evidence that Statway increases student success in acquiring college math credit and enhances equity in student outcomes. Directions for future work are suggested.
Get full access to this article
View all access and purchase options for this article.
References
Austin P. C. (2011). Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies. Pharmaceutical Statistics, 10, 150-161.
Bailey T., Jeong D. W., Cho S. W. (2010). Referral, enrollment, and completion in developmental education sequences in community colleges. Economics of Education Review, 29, 255-270.
Blackwell L. S., Trzesniewski K. H., Dweck C. S. (2007). Implicit theories of intelligence predict achievement across an adolescent transition: A longitudinal study and an intervention. Child Development, 78, 246-263.
Boaler J. (1998). Open and closed mathematics: Student experiences and understandings. Journal for Research in Mathematics Education, 29, 41-62.
Bryk A. S., Gomez L. M., Grunow A. (2011). Getting ideas into action: Building networked improvement communities in education. In Hallinan M. T. (Ed.), Frontiers in sociology of education (pp. 127-162). New York, NY: Springer.
Bryk A. S., Gomez L. M., Grunow A., LeMahieu P. G. (2015). Learning to improve: How America’s schools can get better at getting better. Cambridge, MA: Harvard Education Press.
Carnevale A. P., Desrochers D. M. (2003). Standards for what? The economic roots of K-16 reform. Princeton, NJ: Educational Testing Service.
Cohen G. L., Garcia J., Purdie-Vaughns V., Apfel N., Brzustoski P. (2009). Recursive processes in self-affirmation: Intervening to close the minority achievement gap. Science, 324, 400-403.
Cullinane J., Treisman P. U. (2010, September). Improving developmental mathematics education in community colleges: A prospectus and early progress report on the Statway initiative (An NCPR Working paper). New York, NY: National Center for Postsecondary Research.
Dolle J. R., Gomez L. M., Russell J. L., Bryk A. S. (2013). More than a network: Building communities for educational improvement. In Fishman B. J., Penuel W. R., Allen A.-R., Cheng B. H. (Eds.), Design-based implementation research: Theories, methods, and exemplars. National Society for the Study of Education Yearbook (pp. 443-463). New York, NY: Teachers College Record.
Dweck C. S. (2006). Mindset: The new psychology of success. New York, NY: Random House.
Dweck C. S., Walton G. M., Cohen G. L. (2011). Academic tenacity: Mindsets and skills that promote long-term learning (White paper). Seattle, WA: Gates Foundation.
Edwards A. R., Sandoval C., McNamara H. (2015). Designing for improvement in professional development for community college developmental mathematics faculty. Journal of Teacher Education, 66, 466-481.
Ericsson K. A. (2008). Deliberate practice and acquisition of expert performance: A general overview. Academic Emergency Medicine, 15, 988-994.
Ericsson K. A., Krampe R. T., Tesch-Römer C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100, 363-406.
Gomez K., Gomez L. M., Rodela K. C., Horton E. S., Cunningham J., Ambrocio R. (2015). Embedding language support in developmental mathematics lessons: Exploring the value of design as professional development for community college mathematics instructors. Journal of Teacher Education, 66, 450-465.
Gomez K., Rodela K., Lozano M., Mancevice N. (2013). Designing embedded language and literacy supports for developmental mathematics teaching and learning. MathAMATYC Educator, 5, 43-56.
Grubb W. N. (1999). Honored but invisible: An inside look at teaching in community colleges. New York, NY: Routledge.
Grubb W. N., Gabriner R. (2013). Basic skills education in community colleges: Inside and outside of classrooms. New York, NY: Routledge.
Haynes T. L., Perry R. P., Stupnisky R. H., Daniels L. M. (2009). A review of attributional retraining treatments: Fostering engagement and persistence in vulnerable college students. In Smart J. C. (Ed.), Higher education: Handbook of theory and research (pp. 227-272). New York, NY: Springer.
Hiebert J., Grouws D. (2007). The effects of classroom mathematics teaching on students’ learning. In Lester F. K. Jr. (Ed.), Second handbook of research on mathematics teaching and learning (pp. 371-404). Charlotte, NC: Information Age.
Hodara M. (2013, July). Improving students’ college math readiness: A review of the evidence on postsecondary interventions and reforms (A CAPSEE Working paper). New York, NY: Center for Analysis of Postsecondary Education and Employment.
Hong G., Raudenbush S. W. (2005). Effects of kindergarten retention policy on children’s cognitive growth in reading and mathematics. Educational Evaluation and Policy Analysis, 27, 205-224.
Hong G., Raudenbush S. W. (2006). Evaluating kindergarten retention policy: A case study of causal inference for multilevel observational data. Journal of the American Statistical Association, 101, 901-910.
Jamieson J. P., Mendes W. B., Blackstock E., Schmader T. (2010). Turning the knots in your stomach into bows: Reappraising arousal improves performance on the GRE. Journal of Experimental Social Psychology, 46, 208-212.
Jenkins D., Cho S. W. (2012, January). Get with the program: Accelerating community college students’ entry into and completion of programs of study (CCRC Working Paper No. 32). New York, NY: Teachers College, Community College Research Center, Columbia University.
Ming K., Rosenbaum P. (2000). Substantial gains in bias reduction from matching with a variable number of controls. Biometrics, 56, 118-124.
National Research Council. (2002). Scientific research in education. Washington, DC: National Academy Press.
Pashler H., Rohrer D., Cepeda N. J., Carpenter S. K. (2007). Enhancing learning and retarding forgetting: Choices and consequences. Psychonomic Bulletin & Review, 14, 187-193.
Raudenbush S. W., Bryk A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: SAGE.
Raudenbush S. W., Bryk A. S., Cheong Y. F., Congdon R. T., du Toit M. (2011). HLM 7: Hierarchical linear and nonlinear modeling. Lincolnwood, IL: Scientific Software International.
Rosenbaum P. R., Rubin D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55.
Rosenbaum P. R., Rubin D. B. (1985). Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. The American Statistician, 39, 33-38.
Schmidt R. A., Bjork R. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3, 207-217.
Strother S., Van Campen J., Grunow A. (2013). Community college pathways: 2011-2012 descriptive report. Stanford, CA: Carnegie Foundation for the Advancement of Teaching.
Van Campen J., Sowers N., Strother S. (2013). Community college pathways: 2012-2013 descriptive report. Stanford, CA: Carnegie Foundation for the Advancement of Teaching.
Walton G. M., Cohen G. L. (2011). A brief social-belonging intervention improves academic and health outcomes of minority students. Science, 331, 1447-1451.
Yeager D. S., Walton G. M. (2011). Social-psychological interventions in education: They’re not magic. Review of Educational Research, 81, 267-301.
Biographies
Hiroyuki Yamada is the director of analytics at the Carnegie Foundation for the Advancement of Teaching. He facilitates the development of an analytics agenda that supports the Foundation’s networked improvement communities and leads the design of the analytic strategy and execution of that agenda. His primary interest revolves around practical applications of statistical and measurement models to inform actionable insight for decision making and improvement in education. He holds a Ph.D. in human development and education from UC Berkeley.
Anthony S. Bryk is the ninth president of the Carnegie Foundation for the Advancement of Teaching. He leads work on transforming educational research and development, more closely joining researchers and practitioners to improve teaching and learning. He is also a member of the National Academy of Education. He holds an Ed.D. from Harvard University.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
A Multilevel Model With Propensity Score Matching
Podcast
Cite article
Cite article
Cite article
OR
Download to reference manager
If you have citation software installed, you can download article citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: April 12, 2016
Issue published: July 2016
Keywords
Authors
Metrics and citations
Metrics
Article usage*
Total views and downloads: 869
*Article usage tracking started in December 2016
Articles citing this one
Receive email alerts when this article is cited
Web of Science: 9 view articles Opens in new tab
Crossref: 24
- Network, school, and student demographic level effects of a networked improvement community on student reading outcomes
- Educational outcomes of care leavers and their matched peers: A gender perspective
- Future-forming research in higher education: a review of academic advising literature from the United States
- Status in a psychological statistics class: The role of academic and status-based identities in college students’ subjective social status
- Predicting Educational Attainments and Enrollment to Higher Education: A Longitudinal Study of Care Leavers in Israel
- Effective Learning During COVID-19: Multilevel Covariates Matching and Propensity Score Matching
- Quantifying the Causal Effect of Individual Mobility on Health Status in Urban Space
- Value-Reappraisal and Goal-Setting Intervention Effects on Attitudes and Performance in College Statistics
- Higher Education: Handbook of Theory and Research
- Higher Education: Handbook of Theory and Research
- View More
Figures and tables
Figures & Media
Tables
View Options
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
I am signed in as:
View my profileSign out
I can access personal subscriptions, purchases, paired institutional access and free tools such as favourite journals, email alerts and saved searches.
loading institutional access options
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
