![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
The online administration of student evaluations has its shortcomings, including low participation, or low response rates, and bias. This study examines nonresponse bias in online student evaluations of instruction, that is, the differences between those students who complete online evaluations and those who decide not to complete them. It builds on the work of Estelami that revealed a response bias based on the timing in which the evaluations were completed, that is, differences in early evaluations versus later evaluations. In contrast, this study examines the demographic variables that have contributed to nonresponse bias in online student evaluations, namely gender, grade point average, and ethnicity. It also examines multiple psychographic variables that may contribute to nonresponse bias: time poverty, complaining behavior, and technology savviness. The study utilized t tests and logistic regression (logit analysis) to analyze the data collected from undergraduate business students. This study found that there are significant differences between those who complete online student evaluations and those who do not. Implications for academic administrators are discussed.
References
Section:
|
Adams, M. J. D., Umbach, P. D. (2011). Nonresponse and online student evaluations of teaching: Understanding the influence of salience, fatigue, and academic environments. Research in Higher Education, 53, 576-591. Google Scholar | Crossref | ISI | |
|
Adams, M. J. D., Umbach, P. D. (2012). Nonresponse and online student evaluations of teaching: Understanding the influence of salience, fatigue, and academic environments. Research in Higher Education, 53, 576-591. Google Scholar | Crossref | ISI | |
|
Anderson, D. R., Sweeney, D. J., Williams, T. A. (2009). Essentials of statistics for business and economics (5th ed.). Mason, OH: Thomson Higher Education. Google Scholar | |
|
Anonymous . (2001). Survey profiles young people’s views on community involvement. Fund Raising Management, 32(8), 8. Google Scholar | |
|
Avery, R. J., Bryant, W. K., Mathios, A., Kang, H., Bell, D. (2006). Electronic course evaluations: Does an online delivery system influence student evaluations? Journal of Economic Education, 37, 21-37. Google Scholar | Crossref | ISI | |
|
Baro, E. E., Endouware, B-E. C., Ubogu, J. O. (2011). Information literacy among medical students in the College of Health Sciences in Niger Delta University. Program, 45, 107-120. Google Scholar | Crossref | |
|
Bearden, W. O., Netemeyer, R. G., Haws, K. L. (2011). Handbook of marketing scales: Multi-item measures for marketing and consumer behavior research (3rd ed.). Thousand Oaks, CA: Sage. Google Scholar | |
|
Bladon, T. L. (2010). The downward trend of survey response rates: Implications and considerations for evaluators. Canadian Journal of Program Evaluation, 24, 131-156. Google Scholar | |
|
Braga, M., Paccagnella, M., Pellizzari, M. (2014). Evaluating students’ evaluations of professors. Economics of Education Review, 41(August), 71-88. Google Scholar | Crossref | ISI | |
|
Bruner, G. C., Hensel, P. J., James, K. E. (2005). Marketing scales handbook (Vol. IV). Chicago, IL: American Marketing Association. Google Scholar | |
|
Cheng, W., Ickes, W., Verhofstadt, L. (2012). How is family support related to students’ GPA scores? A longitudinal study. Higher Education, 64, 399-420. Google Scholar | Crossref | ISI | |
|
Clarksberg, M., Robertson, D., Einarson, M. (2008, May). Engagement and student surveys: Nonresponse and implications for reporting survey data. Paper presented at the 48th Annual Forum of the Association for Institutional Research, Seattle, Washington, DC. Google Scholar | |
|
Clayson, D. E. (2009). Student evaluations of teaching: Are they related to what students learn? Journal of Marketing Education, 31, 16-30. Google Scholar | SAGE Journals | |
|
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum. Google Scholar | |
|
Curs, B. R., Harper, C. E. (2012). Financial aid and first-year collegiate GPA: A regression discontinuity approach. Review of Higher Education, 35, 627-649. Google Scholar | Crossref | ISI | |
|
Deviney, D., Mills, L.H., Gerlich, R.N., Santander, C. (2011). Impact of behavioral factors on GPA for gifted and talented students. Academy of Educational Leadership Journal, 15(2), 55-68. Google Scholar | |
|
Dey, E. L. (1997). Working with low survey response rates: the efficacy of weighting adjustments. Research in Higher Education, 38(2), 215-227. Google Scholar | Crossref | ISI | |
|
DiVall, M. V., Hayney, M. S., Marsh, W., Neville, M. W., O’Barr, S. (2013). Perceptions of pharmacy students, faculty members, and administrators on the use of technology in the classroom. American Journal of Pharmaceutical Education, 77(4), 1-7. Google Scholar | Crossref | Medline | ISI | |
|
Diviak, K. R., Wahl, S. K., O’Keefe, J., Mermelstein, R. J., Flay, B. R. (2006). Recruitment and retention of adolescents in a smoking trajectory study: Who participates and lessons learned. Substance Use & Misuse, 41, 175-182. Google Scholar | Crossref | Medline | ISI | |
|
Donovan, J., Mader, C., Shinsky, J. (2007). Online vs. traditional course evaluation formats: Student perceptions. Journal of Interactive Online Learning, 6, 158-180. Google Scholar | |
|
Estelami, H. (2015). The effects of survey timing on student evaluation of teaching (SET) measures obtained using online surveys. Journal of Marketing Education, 37, 54-64. Google Scholar | SAGE Journals | ISI | |
|
Fairfield-Sonn, J. W., Kolluri, B., Singamsetti, R., Wahab, M. (2010). GMAT and other determinants of GPA in an MBA program. American Journal of Business Education, 3, 77-85. Google Scholar | |
|
Feinberg, F. M., Kinnear, T. C., Taylor, J. R. (2013). Modern marketing research (2nd ed.). Mason, OH: South-Western. Google Scholar | |
|
Fidelman, C. G. (2007). Course evaluation surveys: In-class paper surveys versus voluntary online surveys (Doctoral dissertation). Boston, MA: Boston College. (UMI No. 3301790) Google Scholar | |
|
Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70, 646-675. Google Scholar | Crossref | ISI | |
|
Groves, R. M., Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias. Public Opinion Quarterly, 72, 167-189. Google Scholar | Crossref | ISI | |
|
Guder, F., Malliaris, M. (2013). Online course evaluations response rates. American Journal of Business Education, 6, 333-337. Google Scholar | |
|
Gupta, S., Walker, E. D., Swanson, N. J. (2011). Ethical behavior of graduate business students: An examination of the effect of age, gender, GPA, and work experience. Southern Journal of Business and Ethics, 3, 137-151. Google Scholar | |
|
Hamzaee, R. G. (2005). A survey and a theoretical model of distance education programs. International Advances in Economic Research, 11, 215-216. Google Scholar | Crossref | |
|
Hersh, E., Ansolabehere, S. (2013). Gender, race, age and voting: A research note. Politics and Governance, 1, 132-137. Google Scholar | Crossref | |
|
Kim, S. E., Lehto, X. Y. (2011). The voice of tourists with mobility disabilities: Insights from online customer complaint websites. International Journal of Contemporary Hospitality Management, 24, 451-476. Google Scholar | Crossref | ISI | |
|
Link, M. W., Mokdad, A. H., Stackhouse, H. F., Flowers, N. T. (2006). Race, ethnicity, and linguistic isolation as determinants of participation in public health surveillance surveys. Preventing Chronic Disease, 3(1), A09. Google Scholar | |
|
Linse, A. R. (2010). Online student ratings of teaching effectiveness: Analysis of data from select semesters (2009-2010). Report prepared for the Committee on Faculty Affairs of the University Faculty Senate, Pennsylvania State University. Retrieved from http://www.srte.psu.edu/pdf/Online_vs_Paper_Fall2010.pdf Google Scholar | |
|
Lyness, K. S., Kropf, M. B. (2007). Cultural values and potential nonresponse bias: A multilevel examination of cross-national differences in mail survey response rates. Organizational Research Methods, 10, 210-224. Google Scholar | SAGE Journals | ISI | |
|
Madden, T. J., Dillon, W. R., Leak, R. L. (2010). Students’ evaluation of teaching: Concerns of item diagnosticity. Journal of Marketing Education, 32, 264-274. Google Scholar | SAGE Journals | |
|
Malhotra, N. K. (2010). Marketing research: An applied orientation (6th ed.). Boston, MA: Prentice Hall. Google Scholar | |
|
Maroun, N. (2012, September 25). Preparing for the 2020 workplace. Cape Times, p. 4. Google Scholar | |
|
Marquis, M. (2005). Exploring convenience orientation as a food motivation for college students living in residence halls. International Journal of Consumer Studies, 29, 55-63. Google Scholar | Crossref | |
|
Mau, R. R., Opengart, R. A. (2012). Comparing ratings: In-class (paper) versus out of class (online) student evaluations. Higher Education Studies, 2(3), 55-68. Google Scholar | Crossref | |
|
McDaniel, C., Gates, R. (2012). Marketing research (9th ed.). Hoboken, NJ: John Wiley. Google Scholar | |
|
Meng, F., Wang, S., Peters, S., Lawson, G. (2010). A glimpse into the consumer complaining behavior of immigrants. Journal of Consumer Satisfaction, Dissatisfaction, and Complaining Behavior, 23, 111-136. Google Scholar | |
|
Nevo, D., McClean, R., Nevo, S. (2010). Harnessing information technology to improve the process of students’ evaluations of teaching: An exploration of students’ critical success factors of online evaluations. Journal of Information Systems Education, 21(1), 99-109. Google Scholar | |
|
Nunnally, J. C., Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill. Google Scholar | |
|
Packham, G., Jones, P., Miller, C., Brychan, T. (2004). E-learning and retention: Key factors influencing student withdrawal. Education & Training, 46, 335-342. Google Scholar | Crossref | |
|
Parasuraman, A. (2000). Technology Readiness Index (TRI): A multiple-item scale to measure readiness to embrace new technologies. Journal of Service Research, 2, 307-320. Google Scholar | SAGE Journals | |
|
Park, C. N., Son, J-B. (2009). Implementing computer-assisted language learning in the EFL classroom: Teachers’ perceptions and perspectives. International Journal of Pedagogies & Learning, 5(2), 80-101. Google Scholar | Crossref | |
|
Porter, S. R., Umbach, P. D. (2006). Student survey response rates across institutions: Why do they vary? Research in Higher Education, 47, 229-247. Google Scholar | Crossref | ISI | |
|
Pritchard, R. E., Potter, G. C. (2011). Adverse changes in faculty behavior resulting from use of student evaluations of teaching: A case study. Journal of College Teaching & Learning, 8(1), 1-7. Google Scholar | |
|
Pritchard, R. E., Saccucci, M. S., Potter, G. C. (2010). Evaluating a program designed to demonstrate continuous improvement in teaching at an AACSB-accredited college of business at a regional university: A case study. Journal of Education for Business, 85, 280-283. Google Scholar | Crossref | |
|
Reio, T. G. (2007). Survey nonresponse bias in social science research. New Horizons in Adult Education and Human Resource Development, 21, 48-51. Google Scholar | Crossref | |
|
Reynolds, K. E., Beatty, S. E. (1999). A relationship customer typology. Journal of Advertising, 75, 509-523. Google Scholar | |
|
Rudig, W. (2010). Assessing nonresponse bias in activist surveys. Quality & Quantity, 44, 173-180. Google Scholar | Crossref | ISI | |
|
Sax, L. J., Gilmartin, S. K., Lee, J. J., Hagedorn, L. S. (2008). Using web surveys to reach community college students: An analysis of response rates and response bias. Community College Journal of Research and Practice, 32, 712-729. Google Scholar | Crossref | |
|
Scheja, M. (2006). Delayed understanding and staying in phase: Students’ perceptions of their study situation. Higher Education, 52, 421-445. Google Scholar | Crossref | ISI | |
|
Shippee, N. D., Owens, T. J. (2011). GPA, depression, and drinking: A longitudinal comparison of high school boys and girls. Sociological Perspectives, 54, 351-376. Google Scholar | SAGE Journals | ISI | |
|
Smith, S. (2011). Factors that affect the usage of fitness and recreation centers by students on college campuses (Master’s thesis). Retrieved from https://dspace.iup.edu/bitstream/handle/2069/438/Shaina%20Dee%20Smith.pdf?sequence=1 Google Scholar | |
|
Stoll, A. (2010). A qualitative study over low female college student participation in intramural sports (Doctoral dissertation). Available from ProQuest database. (UMI No. 1484660) Google Scholar | |
|
Stormark, K. M., Heiervang, E., Heimann, M., Lundervold, A., Gillberg, C. (2008). Predicting nonresponse bias from teacher ratings of mental health problems in primary school children. Journal of Abnormal Child Psychology, 36, 411-419. Google Scholar | Crossref | Medline | ISI | |
|
Tyler, K. (2008). Generation gaps. HR Magazine, 53(1), 69-72. Google Scholar | |
|
Wei, W. (2010). The influence of self-construal and co-consumption others on consumer complaining behavior (Master’s thesis). West Lafayette, IN: Purdue University. Google Scholar | |
|
Wuensch, K. L. (2014). Binary logistic regression with SPSS. Retrieved from http://core.ecu.edu/psyc/wuenschk/MV/Multreg/Logistic-SPSS.PDF Google Scholar | |
|
Yuksel, A., Kilinc, U. K., Yuksel, F. (2006). Cross-national analysis of hotel customers’ attitudes toward complaining and their complaining behaviours. Tourism Management, 27, 11-24. Google Scholar | Crossref | ISI |
