![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
There is an increasing interest in using longitudinal measures of student achievement to estimate individual teacher effects. Current multivariate models assume each teacher has a single effect on student outcomes that persists undiminished to all future test administrations (complete persistence [CP]) or can diminish with time but remains perfectly correlated (variable persistence [VP]). However, when state assessments do not use a vertical scale or the evolution of the mix of topics present across a sequence of vertically aligned assessments changes as students advance in school, these assumptions of persistence may not be consistent with the achievement data. We develop the “generalized persistence” (GP) model, a Bayesian multivariate model for estimating teacher effects that accommodates longitudinal data that are not vertically scaled by allowing less than perfect correlation of a teacher’s effects across test administrations. We illustrate the model using mathematics assessment data.
References
|
Akaike, H. (1973). Theory and extension of the maximum likelihood principle. In Petrov, B., Csaki, F. (Eds.), Second International Symposium on information theory (pp. 267–281). Budapest: Akademiai Kiado. Google Scholar | |
|
Ballou, D., Sanders, W., Wright, P. (2004). Controlling for student background in value-added assessment of teachers. Journal of Educational and Behavioral Statistics, 29, 37–66. Google Scholar | SAGE Journals | |
|
Barnard, J., McCulloch, R., Meng, X.-L. (2000). Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage. Statistica Sinica, 10, 1281–1311. Google Scholar | |
|
Bifulco, R., Ladd, H. (2004). The impact of charter schools on student achievement: Evidence from North Carolina. Education Finance and Policy, 1, 50–90. Google Scholar | Crossref | |
|
Braun, H. (2005). Using student progress to evaluate teachers: A primer on value-added models (Technical report). Educational Testing Service, Policy Information Center. Google Scholar | |
|
Browne, W. J., Goldstein, H., Rasbash, J. (2001). Multiple membership multiple classification (MMMC) models. Statistical Modelling: An International Journal, 1, 103–124. Google Scholar | SAGE Journals | |
|
Carlin, B., Louis, T. (2000). Bayes and empirical Bayes methods for data analysis. 2nd ed. Boca Raton, FL: Chapman and Hall/CRC Press. Google Scholar | Crossref | |
|
Cornelissen, T. (2008). The stata command felsdvreg to fit a linear model with two high-dimensional fixed effects. Stata Journal, 8, 170–189. Google Scholar | SAGE Journals | |
|
Daniels, M. J. (1999). A prior for the variance in hierarchical models. The Canadian Journal of Statistics, 27, 567–578. Google Scholar | Crossref | |
|
Daniels, M. J., Kass, R. E. (1999). Nonconjugate Bayesian estimation of covariance matrices and its use in hierarchical models. Journal of the American Statistical Association, 94, 1254–1263. Google Scholar | Crossref | |
|
Fitz-Gibbon, C. T. (1997). The value added national project: Final report, feasibility studies for a national system of value added indicators. London: SCAA. Google Scholar | |
|
Gelman, A., Carlin, J., Stern, H., Rubin, D. (1995). Bayesian data analysis. London: Chapman & Hall. Google Scholar | Crossref | |
|
Gelman, A., Rubin, D. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–472. Google Scholar | Crossref | |
|
Gilks, W. R., Richardson, S., Spiegelhalter, D. J. (1996). Markov chain Monte Carlo in practice. London: Chapman & Hall. Google Scholar | |
|
Gill, B., Zimmer, R., Christman, J., Blanc, S.. (2007). State takeover, school restructuring, private management, and student achievement in Philadelphia (MG-533-ANF/WPF/SDP). Santa Monica, CA: RAND. Retrieved from http://www.rand.org/pubs/monographs/MG533 Google Scholar | |
|
Goldhaber, A., Anthony, E.. (2004). Can teacher quality be effectively assessed? Unpublished manuscript. Google Scholar | |
|
Gordon, R., Kane, T., Staiger, D. (2006). Identifying effective teachers using performance on the job (Technical report; White Paper 2006-01). Washington, DC: The Brookings Institution. Google Scholar | |
|
Hamilton, L. S., McCaffrey, D. F., Koretz, D. (2006). Longitudinal and value-added modeling of student performance. In Validating achievement gains in cohort-to-cohort and individual growth-based modeling contexts (pp. 407–434). Maple Grove, MN: JAM Press. Google Scholar | |
|
Harcourt Brace Educational Measurement . (1997). Stanford achievement test series, ninth edition, technical data report. San Antonio, TX: Harcourt Brace and Company. Google Scholar | |
|
Harris, D., Sass, T.. (2006). Value-added models and the measurement of teacher quality. Unpublished manuscript. Google Scholar | |
|
Jacob, B., Lefgren, L.. (2006). When principals rate teachers (Technical Report 2). Education Next. Google Scholar | |
|
Kane, T., Rockoff, J., Staiger, D.. (2006). What does certification tell us about teacher effectiveness? Evidence from New York City. Unpublished manuscript. Google Scholar | Crossref | |
|
Kass, R., Raftery, A. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795. Google Scholar | Crossref | |
|
Le, V., Stecher, B., Lockwood, J., Hamilton, L., Robyn, A., Williams, V. (2006). Improving mathematics and science education: A longitudinal investigation of the relationship between reform-oriented instruction and student achievement (MG-480-EDU). Santa Monica, CA: RAND. Google Scholar | |
|
Lissitz, R. (Ed.). (2005). Value added models in education: Theory and applications. Maple Grove, MN: JAM Press. Google Scholar | |
|
Little, R., Rubin, D. (1987). Statistical analysis with missing data. 2nd ed. New York: John Wiley & Sons. Google Scholar | |
|
Lockwood, J., McCaffrey, D. (2007). Controlling for individual heterogeneity in longitudinal models, with applications to student achievement. Electronic Journal of Statistics, 1, 223–252. Google Scholar | Crossref | |
|
Lockwood, J., McCaffrey, D., Hamilton, L., Stecher, B., Le, V., Martinez, J. (2007). The sensitivity of value-added teacher effect estimates to different mathematics achievement measures. Journal of Educational Measurement, 44, 47–67. Google Scholar | Crossref | |
|
Lockwood, J., McCaffrey, D., Mariano, L., Setodji, C. (2007). Bayesian methods for scalable multivariate value-added assessment. Journal of Educational and Behavioral Statistics, 32, 125–150. Google Scholar | SAGE Journals | |
|
Lunn, D., Thomas, A., Best, N., Spiegelhalter, D. (2000). WinBUGS—A Bayesian modelling framework: Concepts, structure, and extensibility. Statistics and Computing, 10, 325–337. Google Scholar | Crossref | |
|
Martineau, J. (2006). Distorting value added: The use of longitudinal, vertically scaled student achievement data for value-added accountability. Journal of Educational and Behavioral Statistics, 31, 35–62. Google Scholar | SAGE Journals | |
|
McCaffrey, D., Lockwood, J., Koretz, D., Louis, T., Hamilton, L. (2004). Models for value-added modeling of teacher effects. Journal of Educational and Behavioral Statistics, 29, 67–101. Google Scholar | SAGE Journals | |
|
McCaffrey, D., Lockwood, J., Koretz, D., Hamilton, L. (2003). Evaluating value-added models for teacher accountability (MG-158-EDU). Santa Monica, CA: RAND. Google Scholar | |
|
McCaffrey, D., Lockwood, J., Mariano, L., Setodji, C. (2005). Challenges for value-added assessment of teacher effects. In Value added models in education: Theory and applications (pp. 111–144). Maple Grove, MN: JAM Press. Google Scholar | |
|
Rasbash, J., Browne, W. (2001). Modelling non-hierarchical structures. In Leyland, A., Goldstein, H. (Eds.), Multilevel modelling of health statistics (pp. 93–103). Wiley: Google Scholar | |
|
Raudenbush, S. (2004). Schooling, statistics, and poverty: Can we measure school improvement? (Technical report). Princeton, NJ: Educational Testing Service. Google Scholar | |
|
Raudenbush, S., Bryk, A. (2002). Hierarchical linear models: Applications and data analysis methods. 2nd ed. Thousand Oaks, CA: Sage. Google Scholar | |
|
Reckase, M. (2004). The real world is more complicated than we would like. Journal of Educational and Behavioral Statistics, 29, 117–120. Google Scholar | SAGE Journals | |
|
Rothstein, J. . (2008). Teacher quality in educational production: Tracking, decay, and student achievement. Unpublished manuscript, Princeton University. Retrieved November 25, 2008, from http://www.princeton.edu/~jrothst/workingpapers/rothstein_VAM.pdf Google Scholar | |
|
Sanders, W., Saxton, A., Horn, B. (1997). The Tennessee value-added assessment system: A quantitative outcomes-based approach to educational assessment. In Millman, J. (Ed.), Grading teachers, grading schools: Is student achievement a valid evaluational measure? (pp. 137–162). Thousand Oaks, CA: Corwin. Google Scholar | |
|
Schacter, J., Thum, Y. M. (2004). Paying for high and low-quality teaching. Economics of Education Review, 23, 411–430. Google Scholar | Crossref | |
|
Schafer, J. (1997). Analysis of incomplete multivariate data. New York: Chapman & Hall. Google Scholar | Crossref | |
|
Schmidt, H., Houang, R., McKnight, C. (2005). Value-added research: Right idea but wrong solution?. In Lissitz, R. (Ed.), Value added models in education: Theory and applications (pp. 272–297). Maple Grove, MN: JAM Press. Google Scholar | |
|
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461–464. Google Scholar | Crossref | |
|
Searle, S., Casella, G., McCulloch, C. (1992). Variance components. New York: John Wiley & Sons. Google Scholar | Crossref | |
|
Smith, W. B., Hocking, R. R. (1972). Algorithm as 53: Wishart variate generator. Applied Statistics, 21, 341–345. Google Scholar | Crossref | |
|
Spiegelhalter, D., Best, N., Carlin, B., van der Linde, A. (2002). Bayesian measures of model complexity and fit (with discussion). Journal of the Royal Statistical Society Series B, 64, 583–639. Google Scholar | Crossref | |
|
Tanner, M., Wong, W. (1987). The calculation of posterior distributions by data augmentation (with discussion). Journal of the American Statistical Association, 82, 528–550. Google Scholar | Crossref | |
|
van Dyk, D., Meng, X. (2001). The art of data augmentation (with discussion). The Journal of Computational and Graphical Statistics, 10, 1–111. Google Scholar | Crossref | |
|
Zimmer, R., Buddin, R., Chau, D., Daley, G., Gill, B., Guarino, C. (2003). Charter school operations and performance: Evidence from California. Santa Monica, CA: RAND. Google Scholar |
