![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
The validity and reliability of students’ evaluation of teaching effectiveness has been debated since the 1970s. One concern is the extent to which the ratings are influenced by halo, and if so, how does halo affect the interpretation of the ratings? This study seeks to assess the degree to which the halo affects the diagnosticity of individual teaching evaluation items. Statistical methods are used to identify and purge the individual item ratings of halo. Three professors are compared using the observed teaching evaluation scores and the observed scores once the halo has been purged. Results indicate that the halo is present in the scores and that the halo does bias the interpretation of teaching effectiveness, especially when the goal is to compare one professor with another.
|
Abrami, P.C. , Cohen, P.A. , & d’Apollonia, S. (1988). Implementation problems in meta-analysis. Review of Educational Research, 58, 151-179. Google Scholar | SAGE Journals | ISI | |
|
Abrami, P.C. , d’Apollonia, S. , & Cohen, P.A. ( 1990). The validity of student ratings of instruction: What we know and what we do not. Journal of Educational Psychology , 82, 219-231. Google Scholar | Crossref | ISI | |
|
Borman, W.C. ( 1975). Effect of instruction to avoid halo error on reliability and validity of performance evaluation rating. Journal of Applied Psychology, 60, 556-560. Google Scholar | Crossref | ISI | |
|
Borman, W.C. ( 1977). Consistency of rating accuracy and rating errors in the judgement of human performance. Organizational Behavior & Human Performance, 20, 238-252. Google Scholar | Crossref | Medline | ISI | |
|
Boyce, A.C. ( 1915). Method for measuring teachers’ efficiency. In Fourteenth yearbook, Part II of the National Society for the Study of Education (pp. 9-83). Bloomington, IL: Public School Publishing. Google Scholar | |
|
Campbell, D.T. , & Fiske D.W. ( 1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105. Google Scholar | Crossref | Medline | ISI | |
|
Centra, J.A. ( 1993). Reflective faculty evaluation: Enhancing teaching and determining faculty effectiveness. San Francisco, CA: Jossey-Bass. Google Scholar | |
|
Cohen, P.A. ( 1981). Student ratings of instruction and student achievement: A meta-analysis of multisection validity studies. Review of Educational Research, 51, 281-309. Google Scholar | SAGE Journals | ISI | |
|
Clayson, D.E. ( 1989). Halo effects in student evaluation of faculty: A question of validity. Paper presented at "Positioning for the 1990s," Proceedings of the Southern Marketing Association, New Orleans, LA Google Scholar | |
|
Clayson, D.E. ( 1999). Students’ evaluation of teaching effectiveness: Some implications of stability. Journal of Marketing Education , 21, 68-75. Google Scholar | SAGE Journals | |
|
Clayson, D.E. , & Haley, D.A. ( 1990). Student evaluations in marketing: What is actually being measured? Journal of Marketing Education, 12, 9-17. Google Scholar | SAGE Journals | |
|
Clayson, D.E. , & Sheffet, M.J. ( 2006). Personality and the student evaluation of teaching. Journal of Marketing Education, 28, 149-160. Google Scholar | SAGE Journals | |
|
Cooper, W.H. ( 1981). Ubiquitous Halo. Psychological Bulletin , 90, 218-244. Google Scholar | Crossref | ISI | |
|
Cruse, D.A. ( 1987). Student evaluations and the university professor: Caveat professor. Higher Education, 16, 723-737. Google Scholar | Crossref | ISI | |
| d’Apollonia, S. , & Abrami, P.C. ( 1997). Navigating student ratings of instructions. American Psychologist, 52, 1198-1208. Google Scholar | Crossref | ISI | |
|
Dillon, W.R. , Madden, T.J. , Kermani, A. , & Mukherjee, S. ( 2001). Understanding what’s in a brand rating: A model for assessing brand and attribute effects and their relationship to brand equity. Journal of Marketing Research, 38, 415-429. Google Scholar | SAGE Journals | ISI | |
|
Feldman, K.A. ( 1989). The association between student ratings of specific instructional dimensions and student achievement: Refining and extending the synthesis of data from multisection validity studies. Research in Higher Education, 30, 583-645. Google Scholar | Crossref | ISI | |
|
Feldman, K.A. ( 1990). An after word for the association between student ratings of specific instructional dimensions and student achievement: Refining and extending the synthesis of data from multisection validity studies. Research in Higher Education, 31, 315-318. Google Scholar | Crossref | ISI | |
|
Fishbein, M. , & Ajzen, I. ( 1975). Belief, attitude, intention, and behavior: An introduction to theory and research. Reading, MA: Addison Wesley. Google Scholar | |
|
Fisicaro, S.A. , & Lance, C.E. ( 1990). Implications of three causal models for the measurement of halo error. Applied Psychological Measurement, 14, 419-429. Google Scholar | SAGE Journals | ISI | |
|
Greenwald, A.G. ( 1997). Validity concerns and usefulness of student ratings of instruction. American Psychologist, 52, 1182-1186. Google Scholar | Crossref | Medline | ISI | |
|
Greenwald, A.G. , & Gillmore, G.M. (1997). Grading leniency is a removable contaminant of student ratings. American Psychologist, 52, 1209-1217. Google Scholar | Crossref | Medline | ISI | |
|
Marsh, H.W. ( 1987). Students’ evaluations of university teaching: Research findings, methodological issues, and direction for future research. International Journal of Educational Research, 11, 253-387. Google Scholar | Crossref | |
|
Marsh, H.W. , & Dunkin, M.J. ( 1992). Students’ evaluations of university teaching: A multidimensional perspective. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. 8, pp. 143-233). New York, NY : Agathon Press. Google Scholar | |
|
Nisbett, R.E. , & Wilson, T.D. ( 1977). The halo effect: Evidence for unconscious alteration of judgments. Journal of Personality and Social Psychology , 35, 250-256. Google Scholar | Crossref | ISI | |
|
Orsini, J.L. ( 1988). Halo effects in student evaluations of faculty: A case application. Journal of Marketing Education, 10, 38-45. Google Scholar | SAGE Journals | |
|
Pulakos, E.D. , Schmitt, N. , & Ostroff, C. ( 1986). A warning about the use of a standard deviation across dimensions within ratees to measure halo. Journal of Applied Psychology, 71, 29-32. Google Scholar | Crossref | ISI | |
|
Sherman, B.R. , & Blackburn, R.T. (1975). Personal characteristics and teaching effectiveness of college faculty. Journal of Educational Psychology, 67, 124-131. Google Scholar | Crossref | ISI | |
|
Simpson, P.M. , & Siguaw J.A. ( 2000). Student evaluations of teaching: An exploratory study of the faculty response. Journal of Marketing Education, 22, 199-213. Google Scholar | SAGE Journals | |
|
Stumpf, S.A. , & Freedman, R.D. (1979). Expected grade covariation with student ratings of instruction: Individual versus class effects. Journal of Educational Psychology, 71, 293-302. Google Scholar | Crossref | ISI | |
|
Thorndike, E.L. ( 1920). A constant error in psychological ratings. Journal of Applied Psychology, 4, 25-29. Google Scholar | Crossref | |
|
Wells, F.L. ( 1907). A statistical study of literary merit: With remarks on some new phases of the method. Archives of Psychology 7, 5-30. Google Scholar |
