![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
The purpose of this study was to investigate the impact of two different Direct Behavior Rating–Single Item Scale (DBR-SIS) formats on rating accuracy. A total of 119 undergraduate students participated in one of two study conditions, each utilizing a different DBR-SIS scale format: one that included percentage of time anchors on the DBR-SIS scale and an explicit reference to duration of the target behavior (percent group) and one that did not include percentage anchors nor a reference to duration of the target behavior (no percent group). Participants viewed nine brief video clips and rated student behavior using one of the two DBR-SIS formats. Rating accuracy was determined by calculating the absolute difference between participant ratings and two criterion measures: systematic direct observation scores and DBR-SIS expert ratings. Statistically significant differences between groups were found on only two occasions, pertaining to ratings of academically engaged behavior. Limitations and directions for future research are discussed.
Keywords behavior, assessment, direct behavior ratings, accuracy
References
Section:
|
Altmann, J. (1974). Observational study of behavior: Sampling methods. Behaviour, 49, 227–267. Google Scholar | Crossref | Medline | ISI | |
|
Briesch, A. M., Kilgus, S. P., Chafouleas, S. M., Riley-Tillman, T. C., Christ, T. J. (2012). The influence of alternative scale formats on the generalizability of data obtained from Direct Behavior Rating Single-Item Scales (DBR-SIS). Assessment for Effective Intervention, 38, 127–133. doi:10.1177/1534508412441966 Google Scholar | SAGE Journals | |
|
Chafouleas, S. M. (2011). Direct Behavior Rating: A review of the issues and research in its development. Education and Treatment of Children, 34, 575–591. Google Scholar | Crossref | |
|
Chafouleas, S. M., Christ, T., Riley-Tillman, T. C., Briesch, A. M., Chanese, J. (2007). Generalizability and dependability of Direct Behavior Ratings to measure social behavior of preschoolers. School Psychology Review, 36, 63–79. Google Scholar | |
|
Chafouleas, S. M., Kilgus, S. P., Riley-Tillman, T. C., Jaffery, R., Harrison, S. (2012). Preliminary evaluation of various training components on accuracy of Direct Behavior Ratings. Journal of School Psychology, 50, 317–334. doi:10.1016/j.jsp.2011.11.007 Google Scholar | Crossref | Medline | ISI | |
|
Chafouleas, S. M., Riley-Tillman, T. C., Sugai, G. (2007). School-based behavioral assessment: Informing intervention and instruction. New York, NY: Guilford Press. Google Scholar | |
|
Christ, T. J., Boice, C. (2009). A brief review of nomenclature, components, and formatting to inform the development of Direct Behavior Rating (DBR). Assessment for Effective Intervention, 34, 242–250. Google Scholar | SAGE Journals | |
|
Christ, T. J., Riley-Tillman, T. C., Chafouleas, S. M. (2009). Foundation for the development and use of Direct Behavior Rating (DBR) to assess and evaluate student behavior. Assessment for Effective Intervention, 34, 201–213. Google Scholar | SAGE Journals | |
|
Diener, E., Smith, H., Fujita, F. (1995). The personality structure of affect. Journal of Personality and Social Psychology, 50, 130–141. Google Scholar | Crossref | |
|
Grissom, R. J. (1994). Probability of the superior outcome of one treatment over another. Journal of Applied Psychology, 79, 314–316. Google Scholar | Crossref | ISI | |
|
Grissom, R. J., Kim, J. J. (2005). Effect sizes for research: A broad practical approach. Mahwah, NJ: Lawrence Erlbaum. Google Scholar | |
|
Hartley, J., Trueman, A., Rodgers, A. (1984). The effects of verbal and numerical quantifiers on questionnaire responses. Applied Ergonomics, 15, 149–155. Google Scholar | Crossref | Medline | |
|
Jaffery, R., Johnson, A. H., Bowler, M. C., Riley-Tillman, T. C., Chafouleas, S. M., Harrison, S. E. (2015). Using consensus building procedures with expert raters to establish true score estimates of behavior for direct behavior rating. Assessment for Effective Intervention, 40, 195–204. Google Scholar | SAGE Journals | |
|
Keselman, H. J., Algina, J., Lix, L. M., Wilcox, R. R., Deering, K. (2008). A generally robust approach for testing hypotheses and setting confidence intervals for effect sizes. Psychological Methods, 13, 110–129. Google Scholar | Crossref | Medline | |
|
Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22, 1–55. Google Scholar | |
|
Lucas, R. E., Diener, E., Larsen, R. J. (2009). Measuring positive emotions. In Diener, E. (Ed.), Assessing well-being (pp. 139–155). Dordrecht, The Netherlands: Springer. Google Scholar | Crossref | |
|
Newstead, S. E., Arnold, J. (1989). The effect of response format on ratings of teaching. Educational and Psychological Measurement, 49, 33–43. doi:10.1177/0013164489491004. Google Scholar | SAGE Journals | |
|
Nussbaum, E. M. (2014). Categorical and nonparametric data analysis: Choosing the best statistical technique. Florence, KY: Taylor & Francis. Google Scholar | |
|
Podsakoff, P. M., MacKenzie, S. B., Lee, J. Y., Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88, 879–903. Google Scholar | Crossref | Medline | ISI | |
|
Riley-Tillman, T. C., Christ, T. J., Chafouleas, S. M., Boice-Mallach, C. H., Briesch, A. (2011). The impact of observation duration on the accuracy of data obtained from Direct Behavior Rating (DBR). Journal of Positive Behavior Interventions, 13, 119–128. doi:10.1177/1098300710361954 Google Scholar | SAGE Journals | |
|
Rohrmann, B. (2007). Verbal qualifiers for rating scales: Sociolinguistic considerations and psychometric data (Project Report for the University of Melbourne). Retrieved from http://rohrmannresearch.net/pdfs/rohrmann-vqs-report.pdf Google Scholar | |
|
Saal, F. E., Downey, R. G., Lahey, M. A. (1980). Rating the ratings: Assessing the psychometric quality of rating data. Psychological Bulletin, 88, 413–428. Google Scholar | Crossref | ISI | |
|
Saudargas, R., Lentz, F. (1986). Estimating percent of time and rate via direct observation: A suggested observational procedure and format. School Psychology Review, 15, 36–48. Google Scholar | ISI | |
|
Skinner, C. H., Rhymer, K. N., McDaniel, E. C. (2000). Naturalistic direct observation in educational settings. In Shapiro, E. S., Kratochwill, T. R. (Eds.), Conducting school-based assessments of child and adolescent behavior (pp. 21–54). New York, NY: Guilford Press. Google Scholar | |
|
Stevens, S. S. (1946). On the theory of scales of measurement. Science, 103, 677–680. Google Scholar | Crossref | ISI | |
|
Tapp, J. (2002). Multiple Option Observation System for Experimental Studies (MOOSES) [Software]. Retrieved from http://mooses.vueinnovations.com/overview/mooses-overview Google Scholar | |
|
Thurstone, L. L. (1928). Attitudes can be measured. American Journal of Sociology, 33, 529–554. Google Scholar | Crossref | |
|
Tourangeau, R., Rips, L. J., Rasinski, K. (2000). The psychology of survey response. Cambridge, UK: Cambridge University Press. Google Scholar | Crossref |
