![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
Measuring teacher effectiveness is challenging since no direct estimate exists; teacher effectiveness can be measured only indirectly through student responses. Traditional value-added assessment (VAA) models generally attempt to estimate the value that an individual teacher adds to students' knowledge as measured by scores on successive administrations of a standardized test. Such responses, however, do not reflect the long-term contribution of a teacher to real-world student outcomes such as graduation, and cannot be used in most university settings where standardized tests are not given. In this paper, the authors develop a multiresponse approach to VAA models that allows responses to be either continuous or categorical. This approach leads to multidimensional estimates of value added by teachers and allows the correlations among those dimensions to be explored. The authors derive sufficient conditions for maximum likelihood estimators to be consistent and asymptotically normally distributed. The authors then demonstrate how to use SAS software to calculate estimates. The models are applied to university data from 2001 to 2008 on calculus instruction and graduation in a science or engineering field.
References
| Ballou, D., Sanders, W., Wright, P. (2004). Controlling for student background in value-added assessment of teachers. Journal of Educational and Behavioral Statistics, 29, 37–65. Google Scholar | SAGE Journals | |
| Berkhof, J., Snijders, T. A. B. (2001). Variance component testing in multilevel models. Journal of Educational and Behavioral Statistics, 26, 133–152. Google Scholar | SAGE Journals | |
| Breslow, N. E., Clayton, D. G. (1993). Approximate inference in generalized linear mixed models. Journal of the American Statistical Association, 88, 9–25. Google Scholar | |
| Broatch, J. E. (2009). Multivariate models for assessing educational effectiveness with continuous and categorical responses (Unpublished doctoral dissertation). Arizona State University Google Scholar | |
| Corcoran, S. P. (2009). “Value added” measures of teacher quality: Use and policy validity. Paper presented at the NYU Abu Dhabi Conference. Retrieved from https://steinhardt.nyu.edu/scmsAdmin/uploads/002/891/abudhabi2009SC.ppt Google Scholar | |
| Demidenko, E. (2004). Mixed models: Theory and applications. Hoboken, NJ: Wiley. Google Scholar | Crossref | |
| Doran, H. C., Lockwood, J. R. (2006). Fitting value-added models in R. Journal of Educational and Behavorial Statistics, 31, 205–230. Google Scholar | SAGE Journals | |
| Gelman, A. (2006). Prior distributions for variance parameters in hierarchical models. Bayesian Analysis, 1, 515–533. Google Scholar | Crossref | |
| Goldstein, H., Thomas, S. (1996). Using examination results as indicators of school and college performance. Journal of the Royal Statistical Society, Series A, 159, 149–163. Google Scholar | Crossref | |
| Gordon, R., Kane, T. J., Staiger, D. O. (2006). Identifying effective teachers using performance on the job. Washington, DC: The Brookings Institution. Google Scholar | |
| Government Accountability Office . (2006). Science, technology, engineering, and mathematics trends and the role of federal programs (Tech. Rep. No. GAO-06-702T). Washington, DC: Author Google Scholar | |
| Hartley, H. O., Rao, J. N. K. (1967). Maximum-likelihood estimation for the mixed analysis of variance model. Biometrika, 54, 93–108. Google Scholar | Crossref | Medline | |
| Lehmann, E. L. (1999). Elements of large-sample theory. New York, NY: Springer. Google Scholar | Crossref | |
| Lockwood, J. R., McCaffrey, D. F., Mariano, L. T., Setodji, C. (2007). Bayesian methods for scalable multivariate value-added assessment. Journal of Educational and Behavioral Statistics, 32, 125–150. Google Scholar | SAGE Journals | |
| Mardia, K. V., Marshall, R. J. (1984). Maximum likelihood estimation of models for residual covariance in spatial regression. Biometrika, 71, 135–146. Google Scholar | Crossref | |
| Mariano, L. T., McCaffrey, D. F., Lockwood, J. R. (2010). A model for teacher effects from longitudinal data without assuming vertical scaling. Journal of Educational and Behavioral Statistics, 35, 253–279. Google Scholar | SAGE Journals | |
| Martineau, J. A. (2006). Distorting value added: The use of longitudinal, vertically scaled student achievement data for growth-based, value-added accountability. Journal of Educational and Behavorial Statistics, 31, 35–62. Google Scholar | SAGE Journals | |
| McCaffrey, D. F., Lockwood, J. R. (2011). Missing data in value-added modeling of teacher effects. Annals of Applied Statistics. In press, Google Scholar | Crossref | |
| McCaffrey, D. F., Lockwood, J. R., Koretz, D. M., Hamilton, L. S. (2003). Evaluating value-added models for teacher accountability. Santa Monica, CA: Rand Education. Google Scholar | Crossref | |
| McCaffrey, D. F., Lockwood, J. R., Koretz, D., Louis, T. A., Hamilton, L. S. (2004). Models for value-added modeling of teacher effects. Journal of Educational and Behavioral Statistics, 29, 67–101. Google Scholar | SAGE Journals | |
| McCulloch, C. E. (1994). Maximum likelihood variance components estimation for binary data. Journal of the American Statistical Association, 89, 330–335. Google Scholar | Crossref | |
| Miller, J. J. (1977). Asymptotic properties of maximum likelihood estimates in the mixed model of the analysis of variance. The Annals of Statistics, 5, 746–762. Google Scholar | Crossref | |
| National Academy of Sciences . (2007). Rising above the gathering storm: Energizing and employing America for a brighter economic future. Washington, DC: National Academies Press. Google Scholar | |
| National Science Board . (2007). A national action plan for addressing the critical needs of the U.S. science, technology, engineering, and mathematics education system. Arlington, VA: National Science Foundation. Google Scholar | |
| Pinheiro, J. C., Chao, E. C. (2006). Effcient Laplacian and adaptive Gaussian quadrature algorithms for multilevel generalized linear mixed models. Journal of Computational and Graphical Statistics, 15, 58–81. Google Scholar | Crossref | |
| Rabe-Hesketh, S., Skrondal, A. (2001). Parameterization of multivariate random effects models for categorical data. Biometrics, 57, 1256–1264. Google Scholar | Crossref | Medline | |
| Raudenbush, S. W. (2004). What are value-added models estimating and what does this imply for statistical practice?. Journal of Educational and Behavioral Statistics, 29, 121–129. Google Scholar | SAGE Journals | |
| Resnick, E. L. B. (2004). Teachers matter: Evidence from value added assessments. Research Points, 2, 1–4. Google Scholar | |
| Rodríguez, G., Goldman, N. (2001). Improved estimation procedures for multilevel models with binary response: A case-study. Journal of the Royal Statistical Society. Series A, 164, 339–355. Google Scholar | Crossref | |
| Rowan, B., Correnti, R., Miller, R. J. (2002). What large-scale survey research tells us about teacher effects on student achievement: Insights from the Prospects study of elementary schools. Teachers College Record, 104, 1525–1567. Google Scholar | Crossref | |
| Sanders, W. L., Saxton, A. M., Horn, S. P. (1997). The Tennessee value-added multidimensional value added assessment 30 assessment system: A quantitative, outcomes-based approach to educational assessment. In Millman, J. (Ed.), Grading teachers, grading schools: Is student achievement a valid educational measure? (pp. 137–162). Thousand Oaks, CA: Corwin Press. Google Scholar | |
| SAS Institute Inc . (2008). SAS/STAT 9.2 user’s guide. Cary, NC: Author. Google Scholar | |
| Self, S. G., Liang, K.-Y. (1987). Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions. Journal of the American Statistical Association, 82, 605–610. Google Scholar | Crossref | |
| Tekwe, C. D., Carter, R. L., Ma, C.-X., Algina, J., Lucas, M. E., Roth, J., Ariet, M., Fisher, T., Resnick, M. B. (2004). An empirical comparison of statistical models for value-added assessment of school performance. Journal of Educational and Behavioral Statistics, 29, 11–35. Google Scholar | SAGE Journals | |
| U.S. Department of Education . (2009). Students who study science, technology, engineering, and mathematics (STEM) in postsecondary education (Tech. Rep. No. NCES 2009–161). Washington, DC: Author Google Scholar | |
| Verbeke, G., Molenberghs, G. (2000). Linear mixed models for longitudinal data. Secaucus, NJ: Springer-Verlag. Google Scholar | |
| Wolfinger, R., O’Connell, M. (1993). Generalized linear mixed models: A pseudo-likelihood approach. Journal of Statistical Computation and Simulation, 48, 233–243. Google Scholar | Crossref | |
| Wolfinger, R., Tobias, R., Sall, J. (1994). Computing Gaussian likelihoods and their derivatives for general linear mixed models. SIAM Journal on Scientific Computing, 15, 1294–1310. Google Scholar | Crossref |
