![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
To augment traditional lecture with instructional tools that provide options for content representation, learner engagement, and learning expression, we followed the Universal Design for Learning (UDL) principles to design and implement a learning environment for teaching and learning in large lecture classes. To this end, we incorporated four carefully selected instructional tools (PowerPoint, lecture notes, clickers, and MindTap) in the proposed UDL environment for an introductory marketing class of over 600 students. Self-reported and objective measures were collected to assess the effectiveness of the UDL environment by evaluating its impact on perceived learning, satisfaction with the instructional tools, and actual learning. Our study aims to provide educators with suggestions on how to meet the needs of a diverse group of students in large lecture classes without compromising the quality of teaching and learning.
Today’s undergraduate students are increasingly more diverse in background, ability, and learning preferences (American Council on Education, 2005; Levine & Dean, 2012). At the same time, higher education institutions are under pressure to increase institutional productivity by operating with less resources and funding. A larger lecture class is one way to meet the increased productivity goal because a higher student-to-faculty ratio means higher efficiency in utilizing faculty resources (Cuseo, 2007; McDonald, 2013). As a result, it becomes critical to understand how faculty can effectively meet the needs of a large and diverse student population without compromising the quality of teaching and student learning.
The objective of this article is to address these pedagogical issues by creating a learning environment that builds on the Universal Design for Learning (UDL) principles with the goal of providing diverse learners with options for content representation, learner engagement, and learning expression (CAST, 2013; Hall, Meyer, & Rose, 2012; McGuire-Schwartz & Arndt, 2007; Rose & Gravel, 2010). While prior research examined a variety of instructional tools and technological solutions in improving students learning (Eastman, Iyer, & Eastman, 2011; Lincoln, 2008; McCabe & Meuter, 2011; O’Reilly, Rahinel, Foster, & Patterson, 2007), the examination of how these tools can be jointly used and applied in large lecture classes has received limited attention. Furthermore, as instructors can select from a plethora of different instructional tools of various degree of technological sophistication, it is uncertain how these tools can be combined and jointly used both inside and outside of the classroom so that instructional design can be more thoughtfully made and the effectiveness of these tools can be maximized. To address this gap, we propose incorporating four carefully selected instructional tools (PowerPoint, lecture notes, clickers, and MindTap) in a UDL environment that provides students with multiple means of content presentation, engagement, and learning expression. We design and implement the proposed UDL environment in an introductory marketing class of over 600 students, and evaluate its effectiveness by examining the impact of the four instructional tools’ usage on both perceived and actual learning.
We contribute to prior research in three ways. First, we design and implement a UDL environment for teaching and learning in large lecture classes of 600 or more students. Such large lecture classes require special attention as (a) it is difficult to find the right instructional tools and technological solutions to improve learning outcomes while managing hundreds of students at once and (b) the instructor must reach an increasingly diverse student population of different backgrounds, abilities, and learning preferences. Second, we assess the effectiveness of a UDL environment by examining the impact of the use of different instructional tools on both perceived and actual learning. As such, we extend the UDL literature by assessing the effectiveness of UDL beyond perceived learning. By measuring both perceived and actual learning in the same context, we are able to examine the impact of UDL on two learning outcomes: affective learning (positive learning experience) and cognitive learning (actual knowledge gain). This is important as extant UDL literature has primarily focused on assessing affective learning outcomes. By examining the effectiveness of UDL using measures of objective learning outcomes (e.g., exam performance), we succeed in not only overcoming the biases of self-assessed learning but also strengthening the validity of the efficacy of UDL as an educational practice. Third, we provide insights into how much students use each tool, and whether there are differences in usage across students of different backgrounds, majors, and learning preferences. Understanding how much students use each tool is critical as their availability does not necessitate student use. Such insights are important for instructors to make informed decisions on selecting the right tools to create a UDL learning environment for effective teaching and learning in large lecture classes.
Teaching and Learning in Large Lecture Classes
Today’s undergraduate students are becoming increasingly diverse in terms of demographics, income, family structure, immigration status, and attendance patterns, to name a few (American Council on Education, 2005). They also differ in terms of their technological preferences. For instance, prior research suggests that some students prefer more traditional instructional tools for effective engagement, while others prefer more advanced technological options (Buzzard, Crittenden, Crittenden, & McCarty, 2011). Moreover, having grown up in a consumer-driven society, our students prefer choosing the curriculum, content, instructional method, study materials, and the class schedule that best fit their needs (Levine & Dean, 2012). It is a daunting task to satisfy the myriad learning and technological preferences of students in large classes.
Besides having to deal with an increased workload in administrating and managing a diverse body of hundreds of students, instructors of large lecture classes find it difficult to devote attention to individual students or engage them in meaningful discussion, hands-on activities, and active learning exercises (Cuseo, 2007; Exeter et al., 2010; McDonald, 2013; Mulryan-Kyne, 2010). In addition, large auditoriums or theatres with balcony settings make it difficult for students to participate and/or concentrate on learning without being distracted by others. The feeling of anonymity creates impersonal, disengaged, unmotivated, and passive learning environments. Furthermore, being digital natives, our students are regarded as the always connected, social, and tech-savvy Millennials generation (Pew Research Center, 2014). Over 75% of Millennials admit that they cannot live without their laptops or mobile phones and social media defines their lives (Brown, 2011). Their constant need to connect with their social life becomes a major distraction from paying attention in class. As a result, students are prone to behaving disruptively, arriving late and/or leaving early, or not coming to class at all (Cuseo, 2007; Kokkelenberg et al., 2008; Lazear, 2001; Lincoln, 2008; Monks & Schmidt, 2010; Mulryan-Kyne, 2010).
To address these challenges of teaching and learning in large classes, prior research has predominantly focused on the use and implementation of technologies, such as clickers and various course management systems (e.g., Blackboard), that provide instructors with administrative advantage (Camey, Gray, Wert-Gray, 2008; Solano & Mondal, 2012; Sprague & Dahl, 2009; Stanley, 2013; Ueltschy, 2001). However, to accommodate learner differences and variability as well as to resolve pedagogical issues with large lecture classes will require more than selecting and implementing a single technological solution. We attempt to address these challenges by tapping into the promises of UDL to create an inclusive learning environment that reduces barriers to learning and increases access to learning for diverse learners.
Universal Design for Learning
Decades of research in neuroscience and education reveal that there are three basic brain activities in a learning episode: (a) recognition activities to gather and understand information, ideas, and concepts (the what of learning); (b) strategic activities to plan, organize, express ideas, and perform tasks (the how of learning); (c) affective activities to motivate and engage learning (the why of learning; CAST, 2013; Rose & Gravel, 2010). More important, these brain activities are not only unique to each individual but they are also related to the learners’ environment in a complex and dynamic fashion. This means that differences and variability exist not only among individuals but within an individual and need to be addressed in educational practices to provide learning opportunities for all learners (Gargiulo & Metcalf, 2013; Rose, Harbour, Johnston, Daley, & Abarbanell, 2006).
Based on the research in neuroscience and our understanding of the role of brain activities in learning, prior research establishes three UDL principles to address learner differences and variability: (a) provide multiple means of representation (the what of learning), (b) provide multiple means of engagement (the why of learning), and (c) provide multiple means of action and expression (the how of learning; CAST, 2013). These principles are intended to enable instructors to work with diverse populations and to provide access to learning for all students (Hall et al., 2012; McGuire-Schwartz & Arndt, 2007).
Past UDL research in postsecondary education settings has predominantly focused on applying the three principles for instructor training (Davies, Schelly, & Spooner, 2013; McGuire-Schwartz & Arndt, 2007; Schelly, Davies, & Spooner, 2011; Spooner, Baker, Harris, Ahlgrim-Delzell, & Browder, 2007) and web-based/online graduate course design (Rao & Tanners, 2011). Furthermore, its effectiveness was evaluated from the perception of instructors and/or students (Rao, Ok, & Bryant, 2014; Roberts, Park, Brown, & Cook, 2011). For instance, Schelly et al. (2011) examine student perceptions of instruction improvement following UDL training, while Rao and Tanners (2011) study the implementation of UDL principles in an online course, highlighting features of UDL design that students valued. These included, for instance, providing options and choices for student engagement and learning, or incorporating different ways to interact with the class. However, prior research has cautioned about relying solely on perceptual measures for learning outcome assessments as self-reported of learning tend to be biased and do not capture actual knowledge gains (Bacon, 2011; Bowman, 2010; Clayson, 2009; Sitzmann, Ely, Brown, & Bauer, 2010). As a result, we intend to contribute to extending extant UDL literature by examining the effectiveness of UDL using objective learning outcome measures. To the best of our knowledge, we are among the first to investigate how these principles can be incorporated to create a UDL environment in large undergraduate lecture classes and evaluate its effectiveness on both perceived and actual learning.
Proposed UDL Environment for Large Lecture Classes
Following the three principles of UDL, we design and implement a UDL learning environment that augments traditional lecture with various instructional tools to provide options for content representation, learner engagement, and learning expression. All learners are thus given an equal opportunity to learn despite differences and variability in student abilities, background, as well as technological and learning preferences. In the proposed UDL environment, we combine four carefully selected instructional tools to augment the lecture and textbook approach to teaching and learning in large lecture classes. These include clickers for in-class engagement as well as PowerPoint, lecture notes, and MindTap for both inside and outside of class uses.
Instructional Tools
Each lecture is delivered in class via a PowerPoint presentation that is made available to students for download outside of class. Lecture notes, which are also provided to students for download before class, contain a class outline, fill-in-the-blank exercises, and sample exam questions for each lesson (see the appendix). In the proposed UDL environment, we also include two third-party instructional solutions: an audience response system, also known as clickers from Turning Technologies®, and an online learning tool from Cengage Learning called MindTap. While clickers engage students with the material presented in class, MindTap is used to engage students both inside and outside of the classroom. MindTap not only offers students access to their textbooks but it also includes multiple apps (see the appendix) such as flashcards (that allow students to practice memorization of key terms), Merriam–Webster dictionary (for improving students’ understanding of English terms and expressions), and Notebook (for aggregating student annotations and notes). Homework assignments are also administered via MindTap. Both clickers and MindTap are integrated with Blackboard, which not only enhances students’ capacity for monitoring their progress but also provides instructors with administrative advantages. Below, we expand on how these instructional tools can be used to create a learning environment following the three principles of UDL (see Table 1).
|
Table 1. Universal Design for Learning Options for the Three Aspects of Learning.
Means of Representation
In terms of offering multiple means of content representation, the combination of the instructional tools permits students to have the option to listen in class or view a PowerPoint presentation. Students are also given the opportunity to access their textbook in printed, electronic, or audio form through MindTap, offering additional choices in content representation. In this way, students are given alternatives to listen, watch, read, view, and study content in either printed or digital media depending on how, when, and where they prefer accessing the “what” of learning.
Means of Engagement
With respect to learner engagement, each lecture is accompanied by a set of lecture notes. The fill-in-the-blank exercises and sample exam questions that are included in these notes enhance the lecture and engage students with the material. Similarly, clickers are used in each class to administer in-class quizzes and further engage students with the material. Quiz questions are positioned throughout the lecture to break up the content and continuously engage students in class. However, these tools only engage students during class time. To generate opportunities for students to engage with the material outside of the classroom, MindTap offers a variety of apps such as flashcards, dictionary, or notebook. These alternative means of engagement are effectively extending learning opportunities beyond classroom and class time to motivate students to participate in learning activities that reinforce the “why” of learning.
Means of Action and Expression
To provide options for learning expression, fill-in-the-blank exercises from lecture notes and quizzes administered by clickers allow students to gauge where they are in their learning process. Students also use MindTap to complete both before and after lecture homework assignments. Before-lecture homework assignments enable students to come to class already armed with the knowledge they need for the day’s activities. After-lecture homework assignments help students reflect on what they learned in class for deeper learning. Since MindTap is integrated with Blackboard, students can easily monitor their learning progress. These various ways of learning expressions and activities allow students to approach planning, organizing, and performing learning tasks in executing the “how” of learning with a sound learning strategy.
Methodology
The UDL environment was implemented at a large American university for a marketing management class in the Spring and Fall semesters. Both semesters were taught by the same instructor who used identical lectures, materials, and instructional tools. The sample characteristics for both semesters are reported in Table 2.
|
Table 2. Sample Characteristics.
To assess the effectiveness of the UDL environment, we collected data from two sources. First, we administered an in-class survey on the last day of class prior to the final exam. This time frame was selected to ensure full and repeated exposure to all the instructional tools. Across both semesters, we received a total of 928 completed and usable questionnaires out of a total of 1,285, a response rate of 72%. The survey was used to assess students’ satisfaction with the instructional tools, perceived effectiveness, self-reported use of various instructional tools, and perceived learning. To assess potential response bias, we compared those who responded with the survey with those who did not using Pearson chi-square test (when variables were categorical) and difference of means (when variables were continuous). The results reported in Table 2 reveal that there are significant differences across gender and GPA (p < .01). Specifically, we find that survey respondents are more likely to be female with a higher GPA (Porter & Whitcomb, 2005), suggesting that students who did not respond to the survey may be less likely to be engaged in class and thus less likely to use instructional tools. While this finding is consistent with prior research on characteristics of student survey participants, it is important to acknowledge this sample bias when analyzing self-reported data. In addition, prior research (Bacon, 2011; Bowman, 2010; Clayson, 2009; Sitzmann et al., 2010) demonstrates that self-reports of learning and tool usage may not capture actual learning or knowledge gain. For these reasons, collecting objective data, in addition to self-reported measures, is critical in assessing the effectiveness of UDL environment. As such, we used online analytics to capture the actual use of instructional tools and we used exam performance to measure actual learning. Since objective data were available for all students enrolled in the class, we were able to perform the analysis on the full class sample. The sample size for the objective data collection is therefore 1,285.
Measurement
We measured perceived learning, satisfaction with the instructional tools, perceived effectiveness of each instructional tool, and self-reported use of each instructional tool via an in-class survey. We measured actual learning and actual use of instructional tools over the course of the semester using objective data from online analytics. Tables 3 and 4 present correlations and descriptive statistics for key study constructs.
|
Table 3. Correlations and Summary Statistics for Survey Data.
|
Table 4. Correlations and Summary Statistics for Objective Data.
Perceived Learning
We measured student learning with six items (α = .92) using a 5-point Likert-type scale adapted from McCabe and Meuter (2011). Students were asked to rate the extent to which the instructional tools helped them (a) earn a better grade in class, (b) stay interested in the topic of study, (c) retain knowledge long term, (d) enhance their educational experience outside of classroom, (e) enhance their educational experience inside of classroom, (f) and learn more about the topic.
Satisfaction With the Instructional Tools
We used three items (α = .91) each on a 5-point Likert-type scale, adapted from Sprague and Dahl (2009) to operationalize satisfaction with the instructional tools. Students were asked to rate the extent of their agreement with the following statements: (a) I liked the use of different instructional tools in my marketing management class; (b) I believe that by using the instructional tools, my enjoyment of learning about marketing increased; (c) I found that this class was more fun because of the use of the different instructional tools.
Perceived Effectiveness of Each Tool and Self-Reported Use of Each Tool
To assess the perceived effectiveness of each tool, students were asked to rate, on a 5-point semantic differential scale (very ineffective–very effective), how effective each instructional tool was in helping them learn. To assess the self-reported use of each tool, students were asked to indicate how frequently they used each tool on a 3-point scale, ranging from not at all to frequently. The instructional tools included PowerPoint, lecture notes, MindTap, and clickers.
Actual Learning
We assessed actual learning using an average of student scores obtained from three exams that were given throughout the course of the semester (α = .80). Each exam contained 50 multiple choice questions and had a total score of 100 (2 points for each question). The exams were not cumulative. We used the exam scores rather than final grade, because homeworks and quizzes were administered via MindTap and clickers, which could confound our results.
Actual Use of Instructional Tools
The actual use of instructional tools was obtained from online analytics. The use of clickers was measured as the total number of classes in which the student used clickers. Students had the opportunity to use clickers in every class (with the exception of the first three introductory sessions) to remain engaged with learning in class. In total, students had the opportunity to use clickers in 21 lectures. The students’ use of MindTap was measured as the total number of homework exercises that the student completed in MindTap. Students had the option to complete 20 homework exercises throughout the course of the semester, covering all chapters, to remain engaged with learning outside of the classroom. However, since data regarding the use of PowerPoint and lecture notes are not electronically collected, we could not obtain objective measures of these tools. As a result, they were excluded from the analysis of actual learning.
Control Variables
We included five control variables in our analyses: GPA, gender, ethnicity, major, and semester. All of these variables were collected from official records and were used in both the survey and objective data analyses. GPA captures student GPA after the semester ended. Ethnicity captures whether the student is a Caucasian or a minority and major captures whether the student is a marketing or a nonmarketing major. Semester captures whether the class was offered in Spring or Fall.
Results
To evaluate the effectiveness of the UDL environment in large lecture classes, we first examine the results of the survey and subsequently, we report the results of the objective data analysis. In the survey analysis, we compare the perceived effectiveness of each instructional tool to assess whether students perceived that the tools were effective in helping them learn. We also perform a regression analysis to assess how the self-reported use of each tool affects students’ perceived learning. However, since perceptual measures are biased and may not capture actual learning or knowledge gain, we conduct a regression analysis to examine how the actual use of instructional tools affects students’ actual learning. Last, we perform a correlation analysis to compare self-reported tool usage with actual usage, and conduct a series of regressions to examine whether the use of each tool differs across gender, ethnicity, and major. This allows us to test to what extent learner differences and needs of a diverse student population are being met.
Results of the Survey Analysis
Students perceived that each instructional tool was effective in helping them learn. We compared the means for the perceived effectiveness of each tool and used a paired samples t test to assess whether there are significant differences among the means. The results show that all means are significantly different from each other (p < .001), suggesting that students perceived lecture notes (M = 4.5, SD = 0.89) and PowerPoints (M = 4.4, SD = 0.84) as being most effective in helping them learn, followed by clickers (M = 4.2, SD = 0.99), and MindTap (M = 3.9, SD = 1.22).
These results indicate that students may prefer instructor generated content (i.e., lecture notes and PowerPoint) more than third-party materials (i.e., MindTap). Additionally, we found that perceived tool usage enhanced students’ satisfaction with the instructional tools, M = 3.75, t(928) = 46.9, which is significantly higher than the scale midpoint (p < .01).
Perceived Learning
We conducted a regression analysis to test the impact of the self-reported use of each instructional tool (PowerPoints, lecture notes, clickers, and MindTap) on perceived learning, while controlling for GPA, gender, major, ethnicity, and semester. Table 5 shows unstandardized and standardized coefficients and associated t statistics for the model. The adjusted R2 is .11.
|
Table 5. The Impact of Self-Reported Tool Usage on Perceived Learning.
The results of the regression analysis indicate that the self-reported use of PowerPoint has a positive and significant impact on perceived learning (β = .13, p < .001), as does the self-reported use of lecture notes (β = .09, p = .02) and MindTap (β = .10, p = .002). The impact of the self-reported use of clickers on perceived learning was, however, found to be not significant (β = .05, p = .25). Although the differences in effects between PowerPoint and MindTap, F(1, 918) = .72, p = .40, and PowerPoint and lecture notes, F(1, 918) = .64, p = .42, are not statistically significant, the standardized coefficients suggest that self-reported use of PowerPoint may have the strongest impact on perceived learning, followed by self-reported use of MindTap and lecture notes. This is interesting, as it suggests that instructional tools that are accessible both inside and outside the classroom, including PowerPoint, MindTap, and lecture notes are perceived by students as being more effective than entirely in-class tools such as clickers. Conclusive evaluation of the impact of UDL on student learning, however, requires a more objective analysis.
Results of the Objective Data Analysis
We used a regression model to test the influence of the actual use of clickers and MindTap on actual learning during the course of the semester. Specifically, we estimated the following model:
Table 6 shows unstandardized and standardized coefficients and associated t statistics for the model. The adjusted R2 is .478. The results suggest that the actual use of MindTap has a positive and significant impact on actual learning (β = .36, p < .001). To better understand the magnitude of the benefit obtained from the use of Mindtap, it is important to note that our exams had a mean score of 79 and a standard deviation of 10.15. Thus, using the slope coefficients, we estimate that one additional use of MindTap is associated with an average increase of .36 points on the exam. Thus, adding three MindTap experiences could account for a 1-point increase in exam scores. Interestingly, we find that the impact of actual use of clickers on actual learning is not significant (β = −.004, p = .95), which is consistent with our prior findings regarding perceptual measures.
|
Table 6. The Impact of Actual Tool Usage on Actual Learning.
Differences in the Usage of Instructional Tools
Given our findings that highlight the importance of using different instructional tools in helping students learn and reaching a diverse audience, we conducted additional analyses in which we first examine the average tool usage and conduct a correlation analysis in which we compare self-reported use of clickers and MindTap with their actual use. This allows us to further assess differences between perceived and actual measures. Subsequently, we test whether there are any differences in usage across gender, ethnicity, and major (while controlling for GPA).
Table 3 shows the average self-reported use of various instructional tools (using a 3-point scale) obtained from survey. The table shows an overall high usage of tools by students, with clickers reported as being used most frequently (MClicker = 2.76), followed by lecture notes (MNotes = 2.62), MindTap (MMindTap = 2.56), and PowerPoint (MPPT = 2.52). The measures of the actual use of the instructional tools throughout the semester (included in Table 4) show a similar trend, where students participated by using clickers in 16 out of 21 possible lectures, and completed on average 18 out of 20 homework assignments in MindTap. We further compare self-reported use of clickers and MindTap with their actual use in a correlation analysis, shown in Table 7. The results reveal a positive and significant correlation between the self-reported and actual use of clickers (r = .22, p < .001) as well as self-reported and actual use of MindTap (r = .10, p = .002), suggesting that the perceived and actual measures of tool usage correspond. Furthermore, both self-reported and actual use of tools have been found to be highly correlated with their respective learning measures. This finding is consistent with prior correlation research studies on usage of clickers (Camey et al., 2008; Sprague & Dahl, 2009).
|
Table 7. Correlations Between Perceived and Actual Use of Clickers and MindTap.
A series of regression analyses was used to test for differences in the use of various instructional tools across gender, ethnicity, and major, while controlling for GPA. The results, as shown in Table 8, indicate that women report using PowerPoint more than men (β = .24, p < .001). Women were also more likely to use lecture notes (β = .21, p < .001), clickers (β = .11, p < .01 for self-reported and β = .72, p < .001 for actual use), as well as MindTap (β = .21, p < .001 for self-reported and β = .33, p < .01 for actual use) than men. This finding shows that women may not only prefer using a greater variety of tools than men but also that women use each tool more frequently than men. For instance, note that the students actually used, on average, clickers in 16 out of possible 21 lectures. Thus using the slope coefficients, we estimate that women use on average .72 more clickers than men. Interestingly, we found no differences in the self-reported as well as actual use of instructional tools across majors (p > .10). The examination of the impact of ethnicity on actual use of clickers and MindTap reveals that contrary to the impact on self-reported use (which is not significant, p > .10), minorities are less likely to actually use MindTap (β = .36, p = .02), but more likely to use clickers (β = −.46, p < .05). This finding is important as our earlier results suggest that minorities benefit more from the use of clickers than Caucasians.
|
Table 8. Differences in Tool Usage Across Student Characteristics.
Discussion
This research is motivated by three key objectives: (a) to design and implement a UDL environment for teaching and learning in large lecture classes so as to meet the needs of diverse learners without compromising the quality of teaching and learning, (b) to assess the effectiveness of UDL environment by examining the impact of the use of various instructional tools on both perceived and actual learning, and (c) to provide insights into how much students used each tool and whether there were any differences across diverse student populations. Implications from our findings are discussed as follows.
First, by following the UDL principles, we selected four instructional tools (PowerPoint, lecture notes, clickers, and MindTap) to augment traditional lecture and textbook approach to teaching and learning in large lecture classes in such a way that students were given multiple forms of content presentation and delivery, multiple ways of engagement both inside and outside of the classroom, and multiple means to express ideas and demonstrate knowledge gains. Furthermore, the instructional tools were carefully selected so that they could be easily integrated to increase instructor efficiency. For instance, all tools communicate with the course management system (such as Blackboard), enabling interoperability among different learning platforms. Overall, students perceived that the instructional tools were highly effective in helping them learn and that perceived tool usage enhanced students’ satisfaction with the instructional tools. In particular, our results suggest that students may prefer instructor generated content (e.g., lecture notes and PowerPoint) more than third-party materials (e.g., MindTap) as perceived effectiveness for lecture notes and PowerPoint was higher than that of MindTap. In addition, the results suggest that instructional tools that are accessible both inside and outside the classroom (such as MindTap) are more effective than strictly in-class tools (such as clickers) in improving both perceived and actual learning outcomes. These findings thus underscore the importance of engaging students in learning activities both inside and outside of the classroom through the offering of multiple instructional tools that create a UDL environment.
Second, by evaluating the effectiveness of UDL on both perceived and actual learning, we contribute to the UDL literature with an empirical study that extends learning outcome research from the predominately subjective to objective assessments, further strengthening the validity of UDL’s efficacy as an educational practice. Specifically, by examining the impact of the actual use of clickers and MindTap on actual learning over the course of the semester, we find that MindTap has a significant impact on actual learning, whereas no such impact is found with the use of clickers. This finding is consistent with survey results which show that the self-reported use of MindTap has an impact on perceived learning but that clickers do not. These findings suggest that MindTap, which integrates textbook, homework, and learning apps for both inside and outside of classroom access is more effective than in-class engagement tools (e.g., clickers) in improving actual and perceived learning outcomes. Clickers, as an engagement tool, are intended to reinforce understanding of materials discussed in class (Anderson, 2013; Carnaghan, Edmonds, Lechner, & Olds, 2011; Terrion & Aceti, 2012).
We find several differences when comparing survey results with the results of the objective data analysis. The key differences are summarized in Table 9 and discussed as follows. First, the results reveal that students perceived clickers to be more effective in helping them learn than MindTap, and report using clickers more often than MindTap. However, the results using objective measures reveal that MindTap has a stronger impact on actual learning than clickers and that students actually use MindTap more than clickers. Specifically, students used MindTap in 90% of total opportunities, while they used clickers in only 76.2% of classes in which clickers were offered. This finding is interesting as it may suggest that clickers are perceived by students as more salient, potentially because they are used in-class rather than outside of the class. Therefore, if a student is not in class, he or she does not have the opportunity to use a clicker. However, when in attendance, the student might use the clicker in almost every class, creating the perception of a higher usage. However, it is important to note that while MindTap may not be perceived by students as being as effective as clickers, it has a stronger impact on actual learning. This difference seems to suggest that self-reported measures may be capturing other experiences such as satisfaction or positive learning experiences with the tools, rather than actual learning. Second, the table reveals that students with a lower GPA and women perceived a higher level of learning, but the results using objective measures show that higher actual learning was achieved by students with a higher GPA and that gender has no impact on actual learning. Again, these finding suggests that the impact on perceived learning may be driven by student’s learning experiences or satisfaction rather than gains in actual learning. Last, the table reveals that ethnicity has no impact on the self-reported use of instructional tools, but that minorities are less likely to actually use MindTap but more likely to use clickers. Taken together, these differences highlight the importance of using appropriate measures for various learning outcome assessments. In other words, use of self-reported learning may be more appropriate to measure subjective/affective learning outcomes (such as positive learning experiences), whereas objective/cognitive learning outcomes (such as knowledge gains) are better evaluated with actual learning measures (Bacon, 2011; Bowman, 2010; Clayson, 2009; Sitzmann et al., 2010).
|
Table 9. Comparison of the Differences Between Survey and Objective Data Analyses.
Last, we provide insights into how much students use each tool and whether there are any differences in tool usage across diverse student populations. A particularly interesting finding is that, while controlling for GPA, women are more likely to use all instructional tools and use each tool more frequently than men. This may suggest that women prefer using a greater variety of tools than men and that women may prefer to be more engaged with the class material than men. Furthermore, we find that minorities are more likely to actually use clickers, but less likely to actually use MindTap than Caucasians, suggesting that minorities may prefer to engage with in-class tool rather than out of class tools. These findings support the use of various instructional tools to reach an increasingly diverse student audience. Taken together, we provide insights to help instructors make informed UDL design decisions by considering the needs of diverse learners in large lecture classes.
Conclusion
Our research shows the promises of a UDL environment for teaching and learning in large lecture classes. The different options of “what,” “why,” and “how” of learning offered in a UDL environment provide learning opportunities that reach a diverse set of students. As a result, students are empowered to take responsibility for their own learning as barriers so common to learning in a large class setting are decreased in a UDL environment. In addition, instructors of large lecture classes can now have means to improve both perceived and actual learning outcomes. Furthermore, we contribute to extant UDL literature by extending UDL research beyond self-reported learning with the inclusion of actual learning outcome assessments.
While our study was conducted in a lecture class of 600 or more students, our findings should be relevant to classes of 100 to 250 students. However, in order to realize the full potential of teaching large lecture classes in a UDL environment, more research is needed to replicate our study, reexamine the UDL principles, and extend the UDL environment to other contexts such as science, technology, engineering, mathematics disciplines, massive open online classes, and vocational training.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
|
American Council on Education . (2005). College students today: A national portrait. Retrieved from http://www.acenet.edu/news-room/Documents/College-Students-Today-A-National-Portrait-2005.pdf Google Scholar | |
|
Anderson, L. S., Healy, A. F., Kole, J. A., Borune, L. E. (2013). The clicker technique: Cultivating efficient teaching and successful learning. Applied Cognitive Psychology, 27, 222-234. Google Scholar | Crossref | ISI | |
|
Bacon, D. R. (2011). Comparing direct versus indirect measures of the pedagogical effectiveness of team testing. Journal of Marketing Education, 33, 348-358. Google Scholar | SAGE Journals | |
|
Bowman, N. A. (2010). Assessing learning and development among diverse college students. New Directions for Institutional Research, 145, 53-71. Google Scholar | Crossref | |
|
Brown, M. (2011). Connecting with the millennials: A visa study. Retrieved from http://www.visa-asia.com/ap/sea/mediacenter/pressrelease/includes/uploads/Visa_Gen_Y_Report_2012_LR.pdf Google Scholar | |
|
Buzzard, C., Crittenden, V. L., Crittenden, W. F., McCarty, P. (2011). The use of digital technologies in the classroom: A teaching and learning perspective. Journal of Marketing Education, 33, 131-139. Google Scholar | SAGE Journals | |
|
Camey, J. P., Gray, G. T., Wert-Gray, S. (2008). Evaluating the impact of student response systems on student performance. Journal for Advancement of Marketing Education, 13, 1-7. Google Scholar | |
|
Carnaghan, C., Edmonds, T. P., Lechner, T. A., Olds, P. R. (2011). Using student response systems in accounting classroom: Strengths, strategies and limitations. Journal of Accounting Education, 29, 265-283. Google Scholar | Crossref | |
|
CAST . (2013). The three principles of UDL. Retrieved from http://www.udlcenter.org/aboutudl/whatisudl/3principles Google Scholar | |
|
Clayson, D. E. (2009). Student evaluations of teaching: Are they related to what students learn? Journal of Marketing Education, 31, 16-30. Google Scholar | SAGE Journals | |
|
Cuseo, J. (2007). The empirical case against large class size: Adverse effects on the teaching, learning, and retention of first-year students. Journal of Faculty Development, 21(1), 5-21. Google Scholar | |
|
Davies, P. L., Schelly, C. L., Spooner, C. L. (2013). Measuring the effectiveness of universal design for learning intervention in postsecondary education. Journal of Postsecondary Education and Disability, 26, 195-220. Google Scholar | |
|
Eastman, J. K., Iyer, R., Eastman, K. L. (2011). Improving undergraduate student satisfaction with the consumer behavior course: Will interactive technology help? Marketing Education Review, 21, 139-149. Google Scholar | Crossref | |
|
Exeter, D. J., Ameratunga, S., Ratima, M., Morton, S., Dickson, M., Hsu, D., Jackson, R. (2010). Student engagement in very large classes: The teachers’ perspective. Studies in Higher Education, 35, 761-775. Google Scholar | Crossref | ISI | |
|
Gargiulo, R. M., Metcalf, D. (2013). Teaching in today’s inclusive classroom: A universal design for learning approach. Belmont, CA: Cengage Learning. Google Scholar | |
|
Hall, T. E., Meyer, A., Rose, D. H. (2012). Universal design for learning in the classroom: Practical applications. New York, NY: Guilford Press. Google Scholar | |
|
Kokkelenberg, E. C., Dillon, M., Christy, S. M. (2008). The effects of class size on student grades at a public university. Economics of Education Review, 27, 221-233. Google Scholar | Crossref | ISI | |
|
Lazear, E. (2001). Educational production. Quarterly Journal of Economics, 116, 777-803. Google Scholar | Crossref | ISI | |
|
Levine, A., Dean, D. R. (2012). Generation on a tightrope: A portrait of today’s college student. San Francisco, CA: Jossey-Bass. Google Scholar | |
|
Lincoln, D. J. (2008). Teaching with clickers in the large-size principles of marketing class. Marketing Education Review, 18, 39-45. Google Scholar | Crossref | |
|
McCabe, D. B., Meuter, M. L. (2011). A student view of technology in the classroom: Does it enhance the seven principles of good practice in undergraduate education? Journal of Marketing Education, 33, 149-159. Google Scholar | SAGE Journals | |
|
McDonald, G. (2013). Does size matter? The impact of student-staff ratios. Journal of Higher Education Policy and Management, 35, 652-667. Google Scholar | Crossref | |
|
McGuire-Schwartz, M. E., Arndt, J. S. (2007). Transforming universal design for learning in early childhood teacher education from college classroom to early childhood classroom. Journal of Early Childhood Teacher Education, 28, 127-139. Google Scholar | Crossref | |
|
Monks, J., Schmidt, R. (2010). The impact of class size and number of students on outcomes in higher education. Retrieved from http://digitalcommons.ilr.cornell.edu/workingpapers/114/ Google Scholar | |
|
Mulryan-Kyne, C. (2010). Teaching large classes at college and university level: Challenges and opportunities. Teaching in Higher Education, 15, 175-185. Google Scholar | Crossref | ISI | |
|
O’Reilly, N. J., Rahinel, R., Foster, M. K., Patterson, M. (2007). Connecting in megaclasses: The netnographic advantage. Journal of Marketing Education, 29, 69-84. Google Scholar | SAGE Journals | |
|
Pew Research Center . (2014). Millennials in adulthood: Detached from institutions, networked with friends. Retrieved from http://www.dvuli.org/news/millennials-adulthood-detached-institutions-networked-friends Google Scholar | |
|
Porter, S. R., Whitcomb, M. (2005). Non-response in student surveys: The Role of Demographics, Engagement and Personality. Research in Higher Education, 46, 127. Google Scholar | Crossref | ISI | |
|
Rao, K., Ok, M. W., Bryant, B. R. (2014). A review of research on universal design education models. Remedial and Special Education, 35, 153-166. Google Scholar | SAGE Journals | ISI | |
|
Rao, K., Tanners, A. (2011). Curb cuts in cyberspace: Universal instructional design for online courses. Journal of Postsecondary Education and Disability, 24, 211-229. Google Scholar | |
|
Roberts, K. D., Park, H. J., Brown, S., Cook, B. (2011). Universal design for instruction in postsecondary education: A systematic review of empirically based articles. Journal of Postsecondary Education and Disability, 24, 5-15. Google Scholar | |
|
Rose, D. H., Gravel, J. W. (2010). Universal design for learning. In Peterson, P., Baker, E., McGraw, B. (Eds.), International encyclopedia of education (pp. 119-124). Oxford, England: Elsevier. Google Scholar | Crossref | |
|
Rose, D. H., Harbour, W. S., Johnston, C. S., Daley, S. G., Abarbanell, L. (2006). Universal design for learning in postsecondary education: Reflections on principles and their application. Journal of Postsecondary Education and Disability, 19, 135-151. Google Scholar | |
|
Schelly, C. L., Davies, P. L., Spooner, C. L. (2011). Student perceptions of faculty implementation of universal design for learning. Journal of Postsecondary Education and Disability, 24, 17-30. Google Scholar | |
|
Sitzmann, T., Ely, K., Brown, K. G., Bauer, K. N. (2010). Self-assessment of knowledge: A cognitive learning or affective measure? Academy of Management Learning & Education, 9, 169-191. Google Scholar | Crossref | ISI | |
|
Solano, R., Mondal, S. (2012). Using social networks and Internet based audience response systems in the classroom. Pennsylvania Economic Review, 19(1), 60-73. Google Scholar | |
|
Spooner, F., Baker, J. N., Harris, A. A., Ahlgrim-Delzell, L., Browder, D. M. (2007). Effects of training in universal design for learning on lesson plan development. Remedial and Special Education, 28, 108-116. Google Scholar | SAGE Journals | ISI | |
|
Sprague, E. W., Dahl, D. (2009). Learning to click: An evaluation of the personal response system clicker technology in introductory marketing courses. Journal of Marketing Education, 32, 93-103. Google Scholar | SAGE Journals | |
|
Stanley, D. (2013). Can technology improve large class learning? The case of an upper-division business core class. Journal of Education for Business, 88, 265-270. Google Scholar | Crossref | |
|
Terrion, J. L., Aceti, V. (2012). Perceptions of the effects of clicker technology on student learning and engagement: A study of freshmen chemistry students. Research in Learning Technology, 20, 1-11. Google Scholar | |
|
Ueltschy, L. C. (2001). An exploratory study of integrating interactive technology into the marketing curriculum. Journal of Marketing Education, 23, 63-72. Google Scholar | SAGE Journals |