![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
Although the student evaluation of teaching has been extensively researched, no general consensus has been reached about the validity of the process. One contentious issue has been the relationship between the evaluations and learning. If good instruction increases the amount of learning that takes place, then learning and the evaluations should be validly related to each other. A review of the literature shows that attempts to find such a nomological relationship has been complicated by practice, methodology, and interpretation. A meta-analysis of the literature shows that a small average relationship exists between learning and the evaluations but that the association is situational and not applicable to all teachers, academic disciplines, or levels of instruction. It is concluded that the more objectively learning is measured, the less likely it is to be related to the evaluations.
Keywords student evaluation of teaching, learning, grades, rigor, business education
|
Abrami, P.C. , Cohen, P.A. , & d'Apollonia, S. (1988). Implementation problems in meta-analysis. Review of Educational Research, 58(2), 151-179. Google Scholar | SAGE Journals | ISI | |
|
Abrami, P.C. , d'Appollonia, S. , & Cohen, P.A. (1990). Validity of student ratings of instruction: What we know and what we do not. Journal of Educational Psychology , 82, 219-231. Google Scholar | Crossref | ISI | |
|
Adams, J.B. (2005). What makes the grade? Faculty and student perceptions . Teaching of Psychology, 32(1), 21-24. Google Scholar | SAGE Journals | ISI | |
|
Adams, J.V. (1997). Student evaluations: The rating game. Inquiry, 1(2), 10-16. Google Scholar | |
|
Aleamoni, L.M. (1999). Student ratings myths versus research facts from 1924 to 1998. Journal of Personnel Evaluation in Education , 13(2), 153-166. Google Scholar | Crossref | |
|
Attiyeh, R. , & Lumsden, K.G. (1972). Some modern myths in teaching economics: The U. K. experience. American Economic Review, 62, 429-433. Google Scholar | ISI | |
|
Bacon, D.R. , & Novotny, J. (2002). Exploring achievement striving as a moderator of the grading leniency effect. Journal of Marketing Education , 24, 4-14. Google Scholar | SAGE Journals | |
|
Baird, J.S. (1987). Perceived learning in relation to student evaluation of university instruction. Journal of Educational Psychology , 79, 9091. Google Scholar | Crossref | ISI | |
|
Bendig, A.W. (1953). The relationship of level of course achievement to students' instructor and course ratings in introductory psychology. Educational and Psychological Measurement, 13, 437-448. Google Scholar | SAGE Journals | ISI | |
|
Bharadwaj, S. , Futrell, C.M. , & Kantak, D.M. (1993). Using student evaluations to improve learning . Marketing Education Review, 3(2), 16-21. Google Scholar | Crossref | |
|
Birnbaum, M.H. (2000). A survey of faculty opinions concerning student evaluation of teaching. Retrieved June 21, 2008, from http://psych.fullerton.edu/mbirnbaum/faculty3.htm Google Scholar | |
|
Boex, L.F.J. (2000). Attributes of effective economics instructors: An analysis of student evaluations. Research in Economic Education , 31, 211-227. Google Scholar | Crossref | ISI | |
|
Braskamp, L.A. , Caulley, D. , & Costin, F. (1979). Student ratings and instructor self-rating and their relationship to student achievement. American Educational Research Journal, 16, 295-306. Google Scholar | SAGE Journals | ISI | |
|
Braskamp, L.A. , & Ory, J.C. (1994). Assessing faculty work: Enhancing individual and institutional performances. San Francisco: Jossey-Bass. Google Scholar | |
|
Cashin, W.E. (1988). Student ratings of teaching: A summary of the research (IDEA Paper No. 20). Manhattan: Center for Faculty Evaluation & Development, Division of Continuing Education, Kansas State University. Google Scholar | |
|
Cashin, W.E. (1995). Student ratings of teaching: The research revisited (IDEA Paper No. 32). Manhattan: Center for Faculty Evaluation & Development, Division of Continuing Education, Kansas State University. Google Scholar | |
|
Centra, J.A. (1977). Student ratings of instruction and their relationship to student learning. American Educational Research Journal , 14, 17-24. Google Scholar | SAGE Journals | ISI | |
|
Centra, J.A. (2003). Will teachers receive higher student evaluations by giving higher grades and less course work? Research in Higher Education, 44, 495-518. Google Scholar | Crossref | ISI | |
|
Chacko, T.I. (1983). Student ratings of instruction: A function of grading standards. Educational Research Quarterly, 8(2), 19-25. Google Scholar | ISI | |
|
Chonko, L.B. , Tanner, J.F. , & Davis, R. (2002). What are they thinking? Students' expectations and self-assessments. Journal of Education for Business, 77, 271- 281. Google Scholar | Crossref | |
|
Clayson, D.E. (1994). Contrasting results of three methodological approaches on the interpretation of a student evaluation of instruction. In E. W. Chandler (Ed.), Proceedings of the Midwest Marketing Association (pp. 209-214). Chicago: Midwest Marketing Association. Google Scholar | |
|
Clayson, D.E. (2004). A test of the reciprocity effect in the student evaluation of instructors in marketing classes. Marketing Education Review, 14(2), 11-21. Google Scholar | Crossref | |
|
Clayson, D.E. (2005a). Performance overconfidence: Metacognitive effects or misplaced student experience. Journal of Marketing Education , 27, 122-129. Google Scholar | SAGE Journals | |
|
Clayson, D.E. (2005b). Within-class variability in student-teacher evaluations: Example and problems. Decision Sciences Journal of Innovative Education, 3(1), 109-124. Google Scholar | Crossref | |
|
Clayson, D.E. (2007). Conceptual and statistical problems of using between-class data in educational research. Journal of Marketing Education , 29, 34-38. Google Scholar | SAGE Journals | |
|
Clayson, D.E. , Frost, T.F. , & Sheffet, M.J. (2005). Grades and the student evaluation of instruction: A test of the reciprocity effect. Academy of Management Learning & Education, 5(1), 52-65. Google Scholar | Crossref | ISI | |
|
Clayson, D.E. , & Haley, D.A. (1990). Student evaluations in marketing: What is actually being measured? Journal of Marketing Education, 12, 9-17. Google Scholar | SAGE Journals | |
|
Clayson, D.E. , & Sheffet, M.J. (2006). Personality and the student evaluation of teaching . Journal of Marketing Education, 28, 149-160. Google Scholar | SAGE Journals | |
|
Cohen, P.A. (1981). Student ratings of instruction and student achievement: A meta-analysis of multi-section validity studies. Review of Educational Research, 51, 281-309. Google Scholar | SAGE Journals | ISI | |
|
Comm, C.L. , & Manthaisel, D.F.X. (1998). Evaluating teaching effectiveness in America's business schools: Implications for service marketers. Journal of Professional Service Marketing, 16(2), 163-170. Google Scholar | Crossref | |
|
Costin, F. (1978). Do student ratings of college teachers predict student achievement? Teaching of Psychology, 5(2), 86-88. Google Scholar | SAGE Journals | ISI | |
|
Cruse, D.B. (1987). Student evaluations of the university professor: Caveat professor. Higher Education, 16, 723-737. Google Scholar | Crossref | ISI | |
|
Dowell, D.A. , & Neal, J.A. (1982). A selective review of the validity of student ratings of teaching. Journal of Higher Education, 53, 51-62. Google Scholar | ISI | |
|
Doyle, K.O. , & Whitely, S.E. (1974). Student ratings as criteria for effective teaching . American Educational Research Journal, 11, 259-274. Google Scholar | SAGE Journals | ISI | |
|
Faranda, W.T. , & Clarke, I., III. (2004). Student observations of outstanding teaching: Implications for marketing educators. Journal of Marketing Education, 26, 271-281. Google Scholar | SAGE Journals | |
|
Flowers, L. , Osterlind, S.J. , Pascarella, E.T. , & Pierson, C.T. (2001). How much do students learn in college? Journal of Higher Education, 72, 565-583. Google Scholar | ISI | |
|
Frey, P.W. (1973). Student ratings of teaching: Validity of several rating factors. Science, 182(4107), 83-85. Google Scholar | Crossref | Medline | ISI | |
|
Frey, P.W. , Leonard, D.W. , & Beatty, W.W. (1975). Student ratings of instruction: Validation research . American Educational Research Journal, 12, 435-447. Google Scholar | SAGE Journals | ISI | |
|
Gaski, J.F. (1987). On "Construct validity of measures of college teaching effectiveness." Journal of Educational Psychology , 79, 326-330. Google Scholar | Crossref | ISI | |
|
Gaultney, J.F. , & Cann, A. (2001). Grade expectations. Teaching of Psychology, 28(2), 84-87. Google Scholar | SAGE Journals | ISI | |
|
Gillmore, G.M. , & Greenwald, A.G. (1999). Using statistical adjustment to reduce biases in student ratings. American Psychologist , 54, 518-519. Google Scholar | Crossref | ISI | |
|
Goldberg, G. , & Callahan, J. (1991). Objectivity of student evaluations of instructors . Journal of Education for Business, 66, 377-378. Google Scholar | Crossref | |
|
Goldman, L. (1985). The betrayal of the gatekeepers: Grade inflation . Journal of General Education, 37(2), 97-121. Google Scholar | |
|
Gramlich, E.M. , & Greenlee, G.A. (1993). Measuring teaching performance . Research in Economic Education, 24(1), 3-13. Google Scholar | Crossref | ISI | |
|
Greenwald, A.G. (1997). Validity concerns and usefulness of student ratings of instruction. American Psychologist, 52, 1182-1186. Google Scholar | Crossref | Medline | ISI | |
|
Greenwald, A.G. , & Gillmore, G.M. (1997a). Grading leniency is a removable contaminant of student ratings. American Psychologist , 52, 1209-1217. Google Scholar | Crossref | Medline | ISI | |
|
Greenwald, A.G. , & Gillmore, G.M. (1997b). No pain, no gain? The importance of measuring course workload in student ratings of instruction . Journal of Educational Psychology, 89, 743-751. Google Scholar | Crossref | ISI | |
|
Gremler, S.D. , & McCollough, M.A. (2002). Student satisfaction guarantees: An empirical examination of attitudes, antecedents, and consequences. Journal of Marketing Education, 24, 150-160. Google Scholar | SAGE Journals | |
|
Grimes, P.W. (2002). The overconfident principles of economics students: An examination of metacognitive skill. Journal of Economic Education, 33(1), 15-30. Google Scholar | Crossref | ISI | |
|
Guthrie, E.R. (1954). The evaluation of teaching: A progress report . Seattle: University of Washington . Google Scholar | |
|
Hake, R.R. (2002). Problems with student evaluations: Is assessment the remedy? Retrieved June 19, 2007, and February 13, 2008, from http://physics.indiana.edu/~hake/assesstherem1.pdf Google Scholar | |
|
Howard, G.S. , & Maxwell, S.E. (1980). Correlation between student satisfaction and grades: A case of mistaken causation? Journal of Educational Psychology , 72, 810-820. Google Scholar | Crossref | ISI | |
|
Hunter, J.E. , & Schmidt, F.L. (2004). Methods of meta-analysis: Correcting error and bias in research findings. Thousands Oaks, CA: Sage. Google Scholar | Crossref | |
|
Johnson, V.E. (2003). Grade inflation: A crisis in college education . New York: Springer. Google Scholar | |
|
Kaplan, M. , Mets, L.A. , & Cook, C.E. (2000). Questions frequently asked about student ratings forms: Summary of research findings. Retrieved March 31, 2006 , from http://www.crlt.umich.edu/tstrategies/studentratingfaq.html Google Scholar | |
|
Kennedy, E.J. , Lawton, L. , & Plumlee, E.L. (2002). Bliss ignorance: The problem of unrecognized incompetence and academic performance. Journal of Marketing Education, 24, 243-252. Google Scholar | SAGE Journals | |
|
Kolevzon, M.S. (1981). Grade inflation in higher education: A comparative study. Research in Higher Education, 15, 195-212. Google Scholar | Crossref | ISI | |
|
Kulik, J.A. (2001). Student ratings: Validity, utility, and controversy . New Directions for Institutional Research, 109, 9-25. Google Scholar | Crossref | |
|
Laverie, D.A. (2002). Improving teaching through improving evaluation: A guide to course portfolios. Journal of Marketing Education , 24, 104-113. Google Scholar | SAGE Journals | |
|
Lundsten, N.L. (1986). Student evaluations in a business administration curriculum: A marketing viewpoint. AMA Developments in Marketing Science, 9, 169-173. Google Scholar | |
|
Lyons, L.C. (1997). Meta-analysis: Methods of accumulating results across domains. Retrieved February 6, 2008, from http://www.lyons-morris.com/MetaA/index.htm Google Scholar | |
|
Machina, K. (1987). Evaluating student evaluations. Academe, 73(3), 19-22. Google Scholar | Crossref | |
|
Marks, R.B. (2000). Determinants of student evaluations of global measures of instructor and course value. Journal of Marketing Education, 22, 108-119. Google Scholar | SAGE Journals | |
|
Marlin, J.W. , & Niss, J.F. (1980). End-of-course evaluations as indicators of student learning and instructor effectiveness. Journal of Economic Education, 11(2), 16-27. Google Scholar | Crossref | ISI | |
|
Marsh, H.W. (1987). Students' evaluations of university teaching: Research findings, methodological issues, and directions for future research . International Journal of Educational Research, 11, 253-388. Google Scholar | Crossref | |
|
Marsh, H.W. , & Dunkin, M. (1992). Students' evaluations of university teaching: A multidimensional perspective. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. 8, pp. 143-233). New York: Agathon. Google Scholar | |
|
Marsh, H.W. , Hau, K. , Chung, C. , & Siu, T.L. (1997). Students' evaluations of university teaching: Chinese version of the Students' Evaluations of Educational Quality instrument . Journal of Educational Psychology, 89, 568-572. Google Scholar | Crossref | ISI | |
|
Marsh, H.W. , & Roche, L.A. (1997). Making students' evaluations of teaching effectiveness effective. American Psychologist, 52, 1187-1197. Google Scholar | Crossref | ISI | |
|
Marsh, H.W. , & Roche, L.A. (1999). Reply upon SET research. American Psychologist, 54, 517-518. Google Scholar | Crossref | ISI | |
|
Marsh, H.W. , & Roche, L.A. (2000). Effects of grading leniency and low workload on students' evaluations of teaching: Popular myth, bias, validity, or innocent bystanders? Journal of Educational Psychology, 92, 202-228. Google Scholar | Crossref | ISI | |
|
McKeachie, W.J. (1987). Commentary: Instructional evaluation: Current issues and possible improvements. Journal of Higher Education , 58, 344-350. Google Scholar | Crossref | ISI | |
|
Moore, M. , & Trahan, R. (1998). Tenure status and grading practices. Sociological Perspectives, 41, 775-781. Google Scholar | SAGE Journals | ISI | |
|
Moreland, R. , Miller, J. , & Laucka, F. (1981). Academic achievement and self -evaluations of academic performances. Journal of Educational Psychology, 73, 335-344. Google Scholar | Crossref | ISI | |
|
Morley, C. , & Evertt, L. D. (Eds). (1965). Bartlett's familiar quotations . New York: Pocket Books. Google Scholar | |
|
Morsh, J.E. , Burgess, G.G. , & Smith, P.N. (1956). Student achievement as a measure of instructor effectiveness. Journal of Educational Psychology, 47(2), 79-88. Google Scholar | Crossref | ISI | |
|
Palmer, J. , Carliner, G. , & Romer, T. (1978). Leniency, learning, and evaluations. Journal of Educational Psychology, 70, 855-863. Google Scholar | Crossref | ISI | |
|
Paswan, A.K. , & Young, J.A. (2002). Student evaluation of instructors: A nomological investigation using structural equation modeling. Journal of Marketing Education, 24, 193-202. Google Scholar | SAGE Journals | |
|
Pollio, H.R. , & Beck, H.P. (2000). When the tail wags the dog. Journal of Higher Education, 71, 84-102. Google Scholar | ISI | |
|
Powell, R.W. (1977). Grades, learning, and student evaluation of instructors . Research in Higher Education, 7, 193-205. Google Scholar | Crossref | |
|
Redding, R.E. (1998). Students' evaluation of teaching fuel grade inflation . American Psychologist, 53, 1227-1228. Google Scholar | Crossref | ISI | |
|
Remmers, H.H. , & Brandenburg, G.C. (1927). Experimental data on the Purdue Rating Scale for Instruction. Educational Administration and Supervision, 13, 519-527. Google Scholar | |
|
Rodin, M. , & Rodin, B. (1972). Student evaluation of teachers. Science, 177, 1164-1166. Google Scholar | Crossref | Medline | ISI | |
|
Rosenthal, R. , Rosnow, R.I. , & Rubin, D.B. (2000). Contrasts and effect sizes in behavioral research . Cambridge, UK: Cambridge University Press. Google Scholar | |
|
Ryan, J.J. , Anderson, J.A. , & Birchler, A.B. (1980). Student evaluation: The faculty responds. Research in Higher Education, 12, 317-333. Google Scholar | Crossref | ISI | |
|
Schlee, R.P. (2005). Social styles of students and professors: Do students' social styles influence their preferences for professors? Journal of Marketing Education, 27, 130-142. Google Scholar | SAGE Journals | |
|
Schmidt, T.A. , Houston, M.B. , Bettencourt, L.A. , & Boughton, P.D. (2003). The impact of voice and justification on students' perceptions of professors' fairness. Journal of Marketing Education, 25, 177-186. Google Scholar | SAGE Journals | |
|
Schulze, R. (2007). Current methods for meta-analysis: Approaches, issues, and developments. Zeitschrift für Psychologie/Journal of Psychology, 215(2), 90-103. Google Scholar | Crossref | ISI | |
|
Schwab, D.P. (1976). Manual for the Course Evaluation Instrument. Madison: University of Wisconsin, School of Business . Google Scholar | |
|
Scriven, M. (1983). Summative teacher evaluations. In J. Milman (Ed.), Handbook of teacher evaluation (pp. 244-271). Thousand Oaks, CA: Sage. Google Scholar | |
|
Seiver, D.A. (1983). Evaluations and grades: A simultaneous framework . Journal of Economic Education, 14(3), 32-38. Google Scholar | Crossref | ISI | |
|
Seldin, P. (1993, July 21). The use and abuse of student ratings of professors. Chronicles of Higher Education, 39(46), A40. Google Scholar | |
|
Seldin, P. (1999). Changing practices in evaluating teaching: A practical guide to improving faculty performance and promotion/tenure decisions. Bolton, MA: Anker. Google Scholar | |
|
Sheehan, E.P. , & DuPrey, T. (1999). Student evaluations of university teaching. Journal of Instructional Psychology, 26, 188-193. Google Scholar | |
|
Sheets, D.F. , Topping, E.E. , & Hoftyzer, J. (1995). The relationship of student evaluations of faculty to student performance on a common final examination in the principles of economics courses. Journal of Economics, 21(2), 55-64. Google Scholar | |
|
Shmanske, S. (1988). On the measurement of teacher effectiveness. Research in Economic Education, 19, 307-314. Google Scholar | Crossref | ISI | |
|
Simpson, P.M. , & Siguaw, J.A. (2000). Student evaluations of teaching: An exploratory study of the faculty response. Journal of Marketing Education , 22, 199-213. Google Scholar | SAGE Journals | |
|
Sixbury, G.R. , & Cashin, W.E. (1995). Description of database for the IDEA Diagnostic Form (IDEA Technical Report No. 9). Manhattan: Center for Faculty Evaluation & Development, Division of Continuing Education, Kansas State University. Google Scholar | |
|
Soper, J.C. (1973). Soft research on a hard subject: Student evaluations reconsidered. Journal of Economic Research, 5(1), 22-26. Google Scholar | |
|
Stapleton, R.J. , & Murkison, G. (2001). Optimizing the fairness of student evaluations: A study of correlations between instructor excellence, study production, learning production, and expected grades. Journal of Management Education , 25, 269-291. Google Scholar | SAGE Journals | |
|
Steiner, S. , Holley, L.C. , Gerdes, K. , & Campbell, H.E. (2006). Evaluating teaching: Listening to students while acknowledging bias. Journal of Social Work Education, 42, 355-376. Google Scholar | Crossref | ISI | |
|
Stumpf, S.A. , & Freedman, R.D. (1979). Expected grade covariation with student ratings of instruction: Individual versus class effects. Journal of Educational Psychology, 71, 293-302. Google Scholar | Crossref | ISI | |
|
Sullivan, A.M. , & Skanes, G.R. (1974). Validity of student evaluation of teaching and the characteristics of successful instructors. Journal of Educational Psychology, 66, 584-590. Google Scholar | Crossref | ISI | |
|
Theall, M. , & Franklin, J. (2001). Looking for bias in all the wrong places: A search for truth or a witch hunt in student ratings of instruction? New Directions for Institutional Research, 27(5), 45-56. Google Scholar | Crossref | |
|
Weinberg, B.A. , Fleisher, B.M. , & Hashimoto, M. (2007). Evaluating methods for evaluating instruction: The case of higher education (NBER Working Paper No. 12844). Retrieved June 21, 2008, from http://www.nber.org/papers/w12844 Google Scholar | Crossref | |
|
Wilhelm, W.B. (2004). The relative influence of published teaching evaluations and other instructor attributes on course choice. Journal of Marketing Education, 26, 17-30. Google Scholar | SAGE Journals | |
|
Williams, W.M. , & Ceci, S.J. (1997). "How'm I Doing?": Problems with student ratings of instructors and courses. Change, 29(5), 13-23. Google Scholar | Crossref | |
|
Wilson, R. (1998). New research casts doubt on value of student evaluations of professors. Chronicle of Higher Education, 44(19), A12-A14. Google Scholar | |
|
Yunker, P.J. , & Yunker, J. (2003). Are student evaluations of teaching valid? Evidence from an analytical business core course. Journal of Education for Business, 78, 313-317. Google Scholar | Crossref |
