![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
Today, 35 states use performance-based funding models tying appropriations directly to educational outcomes. Financial incentives should induce colleges to improve performance, but there are several well-documented reasons why this is unlikely to occur. We examine how two of the most robust performance funding states—Tennessee and Ohio—responded to the policy. Using a difference-in-differences design, findings point to null and negative effects where colleges responded by producing fewer associate’s degrees or not increasing bachelor’s degree productivity. The only positive and robust effects were found among Tennessee community colleges that responded by producing significantly more certificates. Findings are consistent with performance management literature, where policy impacts are often muted or limited to a narrow range of outcomes.
References
|
Allen, D., Dadgar, M. (2012). Does dual enrollment increase students’ success in college? Evidence from a quasi-experimental analysis of dual enrollment in New York City. New Directions for Higher Education, 2012(158), 11–19. Google Scholar | Crossref | |
|
An, B. P. (2013). The impact of dual enrollment on college degree attainment: Do low-SES students benefit?Educational Evaluation and Policy Analysis, 35(1), 57–75. Google Scholar | SAGE Journals | ISI | |
|
Andrews, M., Moynihan, D. (2002). Why reforms do not always have to “work” to succeed: A tale of two managed competition initiatives. Public Performance & Management Review, 25(3), 282–297. Google Scholar | Crossref | |
|
Angrist, J., Pischke, J. (2015). Mastering ‘metrics: The path from cause to effect. Princeton, NJ: Princeton University Press. Google Scholar | |
|
Bardach, N. S., Wang, J. J., De Leon, S. F., Shih, S. C., Boscardin, W. J., Goldman, L. E., Dudley, R. A. (2013). Effect of pay-for-performance incentives on quality of care in small practices with electronic health records: S randomized trial. JAMA, 310(10), 1051–1059. Google Scholar | Crossref | Medline | ISI | |
|
Belfield, C., Bailey, T. (2017). The labor market returns to sub-baccalaureate college: A review. Retrieved from http://capseecenter.org/labor-market-returns-sub-baccalaureate-college-review/ Google Scholar | |
|
Bertrand, M., Duflo, E., Mullainathan, S. (2004). How much should we trust differences-in-differences estimates?The Quarterly Journal of Economics, 119(1), 249–275. Google Scholar | Crossref | ISI | |
|
Bettinger, E., Baker, R. (2014). The effects of student coaching: An evaluation of a randomized experiment in student advising. Educational Evaluation and Policy Analysis, 36(1), 3–19. Google Scholar | SAGE Journals | ISI | |
|
Bettinger, E., Soliz, A. (2016). Returns to vocational credentials: Evidence from Ohio’s community and technical colleges. Retrieved from https://capseecenter.org/returns-to-vocational-credentials/ Google Scholar | |
|
Bound, J., Lovenheim, M., Turner, S. (2012). Increasing time to baccalaureate degree in the United States. Education Finance and Policy, 7(4), 375–424. Google Scholar | Crossref | ISI | |
|
Callahan, K., Meehan, K., Shaw, K. (2017). Summary of OBF impact on student outcomes in Tennessee and Indiana. Retrieved from https://www.researchforaction.org/publications/summary-obf-impact-student-outcomes-tennessee-indiana/ Google Scholar | |
|
Cancian, M., Haveman, R., Meyer, D., Wolfe, B. (2002). Before and after TANF: The economic well-being of women leaving welfare. Social Service Review, 76(4), 603–641. Google Scholar | Crossref | ISI | |
|
Carey, J. (2014). State share of instruction report. Retrieved from https://www.ohiohighered.org/sites/ohiohighered.org/files/uploads/financial/ssi/SSI_Performance-Based-Funding-Evaluation-Report_Dec2014.pdf Google Scholar | |
|
Castleman, B., Goodman, J. (2016). Intensive college counseling and the enrollment and persistence of low income students. Education Finance and Policy. Retrieved from http://www.mitpressjournals.org/doi/abs/10.1162/EDFP_a_00204 Google Scholar | |
|
Castleman, B. L., Long, B. T. (2016). Looking beyond enrollment: The causal effect of need-based grants on college access, persistence, and graduation. Journal of Labor Economics, 34(4), 1023–1073. Google Scholar | Crossref | ISI | |
|
Conklin, K., Snyder, M., Stanley, J., Boelscher, S. (2016). Rowing together: Aligning state and federal investments in talent to common outcomes. Retrieved from http://www.ecs.org/rowing-together-aligning-state-and-federal-investments-in-talent-to-common-outcomes/ Google Scholar | |
|
Courty, P., Marschke, G. (2004). An empirical investigation of gaming responses to explicit performance incentives. Journal of Labor Economics, 22(1), 23–56. Google Scholar | Crossref | ISI | |
|
Deming, D., Walters, C. (2017). The impacts of price and spending subsidies on US postsecondary attainment (Working paper). Cambridge, MA: NBER. Google Scholar | Crossref | |
|
Dougherty, K., Hong, E. (2006). Performance accountability as imperfect panacea: The community college experience. In Bailey, T., Morest, V. S. (Eds.), Defending the community college agenda (pp. 51–86). Baltimore, MD: Johns Hopkins University Press. Google Scholar | |
|
Dougherty, K., Jones, S. M., Lahr, H., Natow, R. S., Pheatt, L., Reddy, V. (2016a). Looking inside the black box of performance funding for higher education: Policy instruments, organizational obstacles, and intended and unintended impacts. RSF: The Russell Sage Foundation Journal of the Social Sciences, 2(1), 147–173. Google Scholar | Crossref | |
|
Dougherty, K., Jones, S. M., Lahr, H., Natow, R. S., Pheatt, L., Reddy, V. (2016b). Performance funding for higher education. Baltimore, MD: Johns Hopkins University Press. Google Scholar | |
|
Dougherty, K., Natow, R. (2015). The politics of performance funding for higher education: Origins, discontinuations, and transformations. Baltimore, MD: Johns Hopkins University Press. Google Scholar | |
|
Dougherty, K., Reddy, V. (2013). Performance funding for higher education: What are the mechanisms, what are the impacts?ASHE Higher Education Report, 39(2), 1–134. Google Scholar | |
|
Dranove, D., Kessler, D., McClellan, M., Satterthwaite, M. (2003). Is more information better? The effects of “report cards” on health care providers. Journal of Political Economy, 111(3), 555–588. Google Scholar | Crossref | ISI | |
|
Frederickson, H., Smith, K., Larimer, C., Licari, M. J. (2015). The public administration theory primer (3rd ed). Boulder, CO: Westview Press. Google Scholar | |
|
Fung, C., Lim, Y., Mattke, S., Damberg, C., Shekelle, P. (2008). The evidence that publishing patient care performance data improves Quality of care. Annals of Internal Medicine, 148(2), 111–123. Google Scholar | Crossref | Medline | ISI | |
|
Gándara, D., Rippner, J. A., Ness, E. C. (2017). Exploring the “how” in policy diffusion: National intermediary organizations’ roles in facilitating the spread of performance-based funding policies in the States. The Journal of Higher Education, 88, 701–725. Google Scholar | Crossref | ISI | |
|
Gerrish, E. (2016). The impact of performance management on performance in public organizations: A meta-analysis. Public Administration Review, 76, 48–66. Google Scholar | Crossref | ISI | |
|
Gilmour, J. B., Lewis, D. E. (2006). Does performance budgeting work? An examination of the office of management and budget’s PART scores. Public Administration Review, 66(5), 742–752. Google Scholar | Crossref | ISI | |
|
Golden, M. (2000). What motivates bureaucrats?: Politics and administration during the Reagan years. New York, NY: Columbia University Press. Google Scholar | Crossref | |
|
Gormley, W. T., Balla, S. J. (2012). Bureaucracy and democracy: Accountability and performance (3rd ed). Los Angeles, CA: CQ Press. Google Scholar | |
|
Heinrich, C. (2007). False or fitting recognition? The use of high performance bonuses in motivating organizational achievements. Journal of Policy Analysis and Management, 26(2), 281–304. Google Scholar | Crossref | ISI | |
|
Hillman, N., Tandberg, D., Fryar, A. (2015). Evaluating the impacts of “new” performance funding in higher education. Educational Evaluation and Policy Analysis, 37(4), 501–519. Google Scholar | SAGE Journals | ISI | |
|
Hillman, N., Tandberg, D., Gross, J. (2014). Performance funding in higher education: Do financial incentives impact college completions?Journal of Higher Education, 85(6), 826–857. Google Scholar | Crossref | ISI | |
|
Jha, A. K., Joynt, K. E., Orav, E. J., Epstein, A. M. (2012). The long-term effect of premier pay for performance on patient outcomes. New England Journal of Medicine, 366(17), 1606–1615. Google Scholar | Crossref | Medline | ISI | |
|
Johnson, N., Yanagiura, T. (2016). Early results of outcomes-based funding in Tennessee. Retrieved from https://www.luminafoundation.org/resources/early-results-of-outcomes-based-funding-in-tennessee Google Scholar | |
|
Jones, S. (2015). The game changers: Strategies to boost college completion and close attainment gaps. Change: The Magazine of Higher Learning, 47(2), 24–29. Google Scholar | Crossref | |
|
Jones, T., Jones, S., Elliott, K. C., Russell Owens, L., Assalone, A. E., Gándara, D. (2017). Outcomes based funding and race in higher education. New York, NY: Springer International Publishing. Google Scholar | Crossref | |
|
Kaikkonen, D. (2016). Shifting from enrollment- to performance-based funding in higher education: What can we learn from Washington’s experience?Education Finance and Policy, 11(4), 482–498. Google Scholar | Crossref | ISI | |
|
Karp, M. M. (2013). Dual enrollment for college completion: Policy recommendations for Tennessee. New York, NY: Columbia University, Teachers College, Community College Research Center. Google Scholar | |
|
Kuh, G. D., Kinzie, J., Schuh, J. C., Whitt, E. J. Associates. (2010). Student success in college: creating conditions that matter. San Francisco, CA: Jossey-Bass. Google Scholar | |
|
Li, A. Y. (In press). Covet thy neighbor or “reverse policy diffusion”? State adoption of performance funding 2.0. Research in Higher Education. Google Scholar | ISI | |
|
McCubbins, M. D., Noll, R. G., Weingast, B. R. (1987). Administrative procedures as instruments of political control. Journal of Law, Economics, and Organization, 3, 243–277. Google Scholar | |
|
McLendon, M., Hearn, J., Deaton, R. (2006). Called to account: Analyzing the origins and spread of state performance-accountability policies for higher education. Educational Evaluation and Policy Analysis, 28(1), 1–24. Google Scholar | SAGE Journals | ISI | |
|
Miller, T. (2016). Higher education outcomes-based funding models and academic quality. Retrieved from https://www.luminafoundation.org/files/resources/ensuring-quality-1.pdf Google Scholar | |
|
Moynihan, D. P. (2005). Managing for results in an impossible job: Solution or symbol?International Journal of Public Administration, 28(3–4), 213–231. Google Scholar | Crossref | |
|
Moynihan, D. P. (2008). The dynamics of performance management: Constructing information and reform. Washington, DC: Georgetown University Press. Google Scholar | |
|
Ness, E., Deupree, M., Gándara, D. (2014). Campus responses to outcomes-based funding in Tennessee: Robust, aligned, and contested. Retrieved from https://www.tn.gov/assets/entities/thec/attachments/FordFoundationPaper.pdf Google Scholar | |
|
Ohio Board of Regents (2015). Guidelines and procedures for academic program review. Retrieved from: https://www.ohiohighered.org/sites/ohiohighered.org/files/Academic-Program-Review-Guidelines_FINAL_042915.pdf Google Scholar | |
|
Pascarella, E. T., Terenzini, P. T. (2005). How college affects students: A third decade of research. San Francisco, CA: Jossey-Bass. Google Scholar | |
|
Petersen, L. A., Woodard, L. D., Urech, T., Daw, C., Sookanan, S. (2006). Does pay-for-performance improve the quality of health care?Annals of Internal Medicine, 145(4), 265–272. Google Scholar | Crossref | Medline | ISI | |
|
Rabovsky, T. M. (2012). Accountability in higher education: Exploring impacts on state budgets and institutional spending patterns. Journal of Public Administration Research and Theory, 22(4), 675–700. Google Scholar | Crossref | ISI | |
|
Resh, W. G., Pitts, D. W. (2013). No solutions, only trade-offs? Evidence about goal conflict in street-level bureaucracies. Public Administration Review, 73(1), 132–142. Google Scholar | Crossref | ISI | |
|
Rutherford, A., Rabovsky, T. (2014). Evaluating impacts of performance funding policies on student outcomes in higher education. The ANNALS of the American Academy of Political and Social Science, 655(1), 185–208. Google Scholar | SAGE Journals | ISI | |
|
Sanford, T., Hunter, J. M. (2011). Impact of performance-funding on retention and graduation rates. Education Policy Analysis Archives, 19(33), 1–30. Google Scholar | |
|
Scrivener, S., Weiss, M. J., Ratledge, A., Rudd, T., Sommo, C., Fresques, H. (2015). Doubling graduation rates: Three-year effects of CUNY’s Accelerated Study in Associate Programs (ASAP) for developmental education students. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2571456 Google Scholar | |
|
Shen, Y. (2003). Selection incentives in a performance-based contracting system. Health Services Research, 38(2), 535–552. Google Scholar | Crossref | Medline | ISI | |
|
Shin, J. C. (2010). Impacts of performance-based accountability on institutional performance in the US. Higher Education, 60(1), 47–68. Google Scholar | Crossref | ISI | |
|
Shin, J.-C., Milton, S. (2004). The effects of performance budgeting and funding programs on graduation rate in public four-year colleges and universities. Education Policy Analysis Archives, 12(22). Retrieved from http://epaa.asu.edu/ojs/article/view/177 Google Scholar | |
|
Snyder, M. (2015). Driving better outcomes: Typology and principles to inform outcomes-based funding models. Washington, DC: HCM Strategists. Google Scholar | |
|
St. Clair, T., Cook, T. D. (2015). Difference-in-differences methods in public finance. National Tax Journal, 68(2), 319–338. Google Scholar | Crossref | ISI | |
|
St. Clair, T., Cook, T. D., Hallberg, K. (2014). Examining the internal validity and statistical precision of the comparative interrupted time series design by comparison with a randomized experiment. American Journal of Evaluation, 35(3), 311–327. Google Scholar | SAGE Journals | ISI | |
|
Tandberg, D., Hillman, N. (2014). State higher education performance funding: Data, outcomes and policy implications. Journal of Education Finance, 39(1), 222–243. Google Scholar | |
|
Tandberg, D., Hillman, N., Barakat, M. (2014). State higher education performance funding for community colleges: Diverse effects and policy implications. Teacher’s College Record, 116(12), 1–31. Google Scholar | Medline | ISI | |
|
Tennessee Board of Regents . (2015). Approval of academic programs, units, and modifications. Retrieved from https://policies.tbr.edu/policies/approval-academic-programs-units-and-modifications Google Scholar | |
|
Tennessee Higher Education Commission . (2016). Profiles and trends in Tennessee higher education. Retrieved from https://www.tn.gov/assets/entities/thec/attachments/Profiles-Trends-2016.pdf Google Scholar | |
|
Tennessee Higher Education Commission . (2017). Formula weighted outcomes calculation. Retrieved from https://www.tn.gov/thec/article/2015-20-funding-formula Google Scholar | |
|
Tinto, V. (2012). Completing college: Rethinking institutional action. Chicago, IL: University of Chicago Press. Google Scholar | Crossref | |
|
Umbricht, M. R., Fernandez, F., Ortagus, J. C. (2017). An examination of the (un) intended consequences of performance funding in higher education. Educational Policy, 31, 643–673. Google Scholar | SAGE Journals | ISI | |
|
Volkwein, J. F., Tandberg, D. A. (2008). Measuring up: Examining the connections among state structural characteristics, regulatory practices, and performance. Research in Higher Education, 49(2), 180–197. Google Scholar | Crossref | ISI | |
|
Wachter, R. M., Flanders, S. A., Fee, C., Pronovost, P. J. (2008). Public reporting of antibiotic timing in patients with pneumonia: Lessons from a flawed performance measure. Annals of Internal Medicine, 149(1), 29–32. Google Scholar | Crossref | Medline | ISI | |
|
Weibel, A., Rost, K., Osterloh, M. (2010). Pay for performance in the public sector—Benefits and (hidden) costs. Journal of Public Administration Research and Theory, 20(2), 387–412. Google Scholar | Crossref | ISI | |
|
Welker, J. A., Huston, M., McCue, J. D. (2008). Antibiotic timing and errors in diagnosing pneumonia. Archives of Internal Medicine, 168(4), 351–356. Google Scholar | Crossref | Medline | |
|
Werner, R. M., Asch, D. A. (2005). The unintended consequences of publicly reporting quality information. Journal of the American Medical Association, 293(10), 1239–1244. Google Scholar | Crossref | Medline | ISI | |
|
Werner, R. M., Kolstad, J. T., Stuart, E. A., Polsky, D. (2011). The effect of pay-for-performance in hospitals: Lessons for quality improvement. Health Affairs, 30(4), 690–698. Google Scholar | Crossref | ISI | |
|
Wittman, M. D. (2014). Public funding of airport incentives in the United States: The efficacy of the Small Community Air Service Development Grant program. Transport Policy, 35, 220–228. Google Scholar | Crossref | ISI | |
|
Wood, B. D., Waterman, R. W. (1994). Bureaucratic dynamics: The role of bureaucracy in a democracy. Boulder, CO: Westview Press. Google Scholar | |
|
Zumeta, W., Li, A. Y. (2016). Assessing the underpinnings of performance funding 2.0: Will this dog hunt. Retrieved from https://www.tiaainstitute.org/sites/default/files/presentations/2017-02/ti_assessing_the_underpinnings_of_performance_funding_2.pdf Google Scholar |
