The purpose of this study was to explore the effect of problem-based learning (PBL) on student achievement and students’ perceptions of classroom quality. A group of students taught using PBL and a comparison group of students taught using traditional instruction were studied. A total of 457 students participated in the study. Pre- and post-student achievement data were collected using a 25-item multiple-choice test that aligned with state and local objectives. Data analysis indicated statistically significant gain scores in both the groups with a higher gain score in the PBL group. Data analysis also revealed statistically significant differences in the total score on the Student Perceptions of Classroom Quality (SPOCQ) in favor of the PBL group. This study found positive effects for well-implemented PBL instruction with these students. Future research should include longitudinal studies expanded to different subjects, grade levels, and populations of students.

One of the challenges facing educators of gifted students is the increasing expectation that they prepare students for both college and career by developing skills they will need to thrive in the 21st century. However, meeting this expectation successfully will require shifts in instructional practices, at the least. For example, Zsiga and Webster (2007) note how recent sociological and economic trends, such as overseas competition, new technology, and the increasing complexity of the workplace, influence the needs and future success of students in the 21st century increasing the importance of practices such as self-directed learning. Therefore, the success of these shifts hinges on the effectiveness of promoting practices fundamentally different from the traditional methods of direct instruction to which teachers have reverted as they strived to meet contemporary accountability expectations (Abrams, Pedulla, & Madaus, 2003; Brown, Avery, VanTassel-Baska, Worley, & Stambaugh, 2006; Moon, Brighton, & Callahan, 2003). As Reis et al. (1993) argued, over two decades ago, gifted students in particular, who often come to school with a greater depth of knowledge and a greater capacity to acquire and master content rapidly, often suffer from this approach.

In particular, concrete answers to questions, including what practices prepare teachers to support initiatives such as the Common Core State Standards (CCSS), and what practices prepare students, in particular gifted students, for college and career readiness and to develop 21st-century skills, are necessary to address. To address the question of what practices prepare teachers’ need to teach the new standards and skills, this study drew upon the literature from problem-based learning (PBL) in medical education. PBL is a constructivist instructional model that originated in medical schools (Barrows, 1994) and has been studied extensively in the medical school literature. It has emerged as one practice in the repertoire of current teaching practices accessible in the general curriculum of schools with the potential to provide gifted students with the skills, content, and dispositions necessary for the rigor of the new CCSS standards, college and career readiness expectations, and 21st-century skills.

However, regardless of gifted students’ greater depth of knowledge and a greater capacity to acquire and master content rapidly, common concerns articulated by teachers when considering an indirect, constructivist teaching model such as PBL is that a self-directed, student-centered learning environment (a) takes too much time to plan and implement, (b) does not provide scaffold for students who do not have the requisite prior knowledge or skills to be successful, and (c) is not conducive to students acquiring the required content because it is not explicitly taught (Ertmer & Simons, 2006).

Thus, the question becomes, how well does learning middle school science in a PBL setting compare with learning the same material in a traditional, teacher-centered classroom for gifted students?

The development of talent is a lifelong process, but the middle school years are particularly important. Evidence suggests that gifted students make crucial adolescent developmental transitions earlier than their typically developing peers, including the acquisition of abstract formal operational reasoning and expansion of openness to experience (Gallagher, 2009, 2012). Differentiated education for gifted students during this time period should target these capacities, as well as their ability to learn rapidly. Differentiated educational experiences consist of adjustments in the level, depth, and pacing of curriculum, and that may require additional interventions that may not be routinely practiced in typical classrooms. Well-accepted long-standing guidelines suggest that high-quality curriculum for gifted learners should

  • integrate disciplinary or interdisciplinary content using concepts as organizing foci;

  • use abstraction, depth, breadth, and complexity to advance levels of understanding beyond the general education curriculum;

  • emphasize real-world problems;

  • ask students to function as practicing professionals by using processes and materials that approximate those of an expert or disciplinarian;

  • allow self-directed learning guided by student interests;

  • support flexibility in pacing, and variety;

  • result in authentic products where students transform knowledge in ways that demonstrate a shift in perspective through reinterpretation or extension, preferably including conceptual reasoning (Gallagher, 2009; Hockett, 2009; Maker & Nielson, 1995; Siegle, Wilson, & Little, 2013; VanTassel-Baska & Stambaugh, 2006).

In an analysis of the efficacy of curriculum models for gifted education, VanTassel-Baska and Brown (2007) report that six of the models examined show evidence of effectiveness; most of those favored an inquiry-based model of instruction. Central to the inquiry-based approach was engaging students in asking questions and pursuing answers that involved complex problem solving and decision making. Not all problems are created equally, though. Maker and Schiever (1991) present a problem continuum that consists of six problem types varied in complexity based on whether the structure, method of solution, and solution are known to the learner, the teacher, or both. As the number of unknowns in a problem increases, to either teacher or learner, so does the complexity of the problem. The more complex problems in their matrix satisfy the gifted student’s need for challenge and seemingly fulfill his or her motivation to learn.

PBL is built heavily upon the work of educational psychologist Jerome Bruner (1962) whose educational philosophies had a foundation in discovery through problem solving, a constructivist practice. Bruner’s approach to teaching as discovery extended the work of John Dewey, whose perspective includes experiential learning. Savery and Duffy (1995) characterized the philosophical view of constructivist learning in terms of the following three primary propositions: (a) understanding is in our interactions with the environment; (b) cognitive conflict or puzzlement is the stimulus for learning, and determines the organization and nature of what is learned; and (c) knowledge evolves through social negotiation and through the evaluation of the viability of individual understandings. The core concept of constructivism is that which we understand is a function of the context of the learner. Cognition is a part of the learner’s context. In addition, in this view, learning begins with a puzzlement. The learner’s curiosity about the puzzlement is the stimulus for learning, and the puzzlement is the purpose for undertaking the learning process.

As researchers have noted, PBL is, most importantly, a student-centered, not a teacher-centered method of instruction (Ertmer, & Simons, 2006; Hitchcock & Mylona, 2000; Lehman, George, Buchanan, & Rush, 2006; Wetzel, 1996). According to Wetzel (1996), during PBL students are expected to raise questions about an ill-structured problem, the development of which springs from the content in the curriculum, and then to propose hypotheses, present data from independent study, set and prioritize the study agenda, and work in groups to question and teach each other.

PBL as a method has been studied more widely in medical schools than in P-12 education. More than 80% of the medical schools in the United States use PBL as a primary instructional strategy (Jonas, Etzel, & Barzansky, 1989). In a meta-synthesis approach, Strobel and van Barneveld (2009) analyzed eight meta-analyses of PBL. Of the eight meta-analyses in their study, seven focused on medical education and one focused on tertiary education. Results showed PBL to be superior in terms of long-term retention, skill development, and satisfaction of students and teachers. Conversely, traditional approaches were more effective for short-term retention.

While it is not widely practiced within the context of P-12 education, Savery (2006) identifies three characteristics of PBL that clearly transfer to that setting: (a) the role of the teacher as a facilitator of learning, (b) the responsibilities of the learners to be self-directed and self-regulated in their learning, and (c) the essential elements in the design of ill-structured instructional problems as the driving force for inquiry.

Given these characteristics, Stepien and Pyke (1997) adapted the model used in medical schools for P-12 education. Their model includes five phases: (a) Problem Engagement, (b) Inquiry and Investigation, (c) Problem Definition, (d) Problem Resolution, and (e) Problem Debriefing. In addition, Gallagher (2001) distinguishes three key defining features of PBL instruction: (a) the ill-structured problem, (b) the teacher serves as a meta-cognitive coach, and (c) the student is the primary stakeholder in the learning process.

While traditional classroom instruction may at times include elements of Gallagher’s (2001) three key defining features, PBL uses all of them concurrently throughout the duration of the PBL curriculum phases. These instructional features often pose a challenge for teachers in implementing the model with fidelity as they require the teacher to function in a way that is either unfamiliar or inconsistent with his or her typical practice. Well-designed professional development helps the teacher transition to implement PBL, so that these three key features occur in concordance with each other. The consistent and balanced application of these instructional components is critical to the success and fidelity of implementing PBL.

PBL was designed to create change simultaneously in curriculum and instruction (Barrows, 1988; Barrows & Tamblyn, 1980). Although research in P-12 has been sparse, it is rapidly growing, and there are some indicators of the strengths of PBL and how to maximize its effectiveness in both cognitive and non-cognitive domains.

Cognitive Outcomes

In terms of cognitive outcomes for P-12, a body of research that shows the effects of PBL is consistent with 21st-century skills and centers on content acquisition, conceptual understanding, questioning and problem-solving skills. Included in this is the extensive work of Shelagh Gallagher, which focuses, among other things, on the effects of PBL on the achievement and engagement of gifted students in particular. Regarding content acquisition, the literature consistently shows that students, including gifted students, who participate in PBL instruction have the same or better content acquisition as students taught via traditional methods and, as an added benefit, have longer term retention (Diggs, 1997; Gallagher & Stepien, 1996). The results regarding conceptual understanding further elucidate the effectiveness of PBL in terms of cognitive outcomes. Studies in this area show that PBL increased conceptual knowledge, and promoted depth and complexity of understanding of concepts for students (Dods, 1997; Hmelo, Holton, Allen, & Kolodner, 1997). Additional results include PBL students, including gifted students, generating more perspectives on the problem, defining a higher number of concepts and a more systemic understanding indicating significantly better acquisition of scientific concepts (Gallagher & Stepien, 1996; Hmelo et al., 1997; Sungur, Tekkaya, & Geban, 2006; Wirkala & Kuhn, 2011).

Skills such as problem solving, questioning, and data literacy emerge from the results of the literature as a benefit of PBL. These outcomes are consistent with 21st-century-learning skills. Two studies found that students, including gifted students, obtain problem-solving skills, specifically, problem finding as a step in problem solving, and that the students were able to transfer them to other work (Gallagher, Stepien, & Rosenthal, 1992; Pedersen & Liu, 2003). In terms of questioning, results showed that students developed questioning skills and inquiry capacity such as posing active inquiry questions and generating hypothesis-driven approaches to inquiry in their questions in PBL settings better than in traditional settings (Kang, DeChenne, & Smith, 2012; Zhang, Parker, Eberhardt, & Passalacqua, 2011). Furthermore, previous literature reported that the ability to ask a relevant and authentic question was important to sustaining students’ interest in the project and drove students’ course of learning and inquiry (Chin & Chia, 2004, 2006). In addition, PBL has been shown to support the development of data literacy skills, the ability to make sense of information by collecting and analyzing information, an increasingly important skill in our modern lives (Swan et al., 2013).

Non-Cognitive Outcomes

The focus of PBL studies is not limited to cognitive outcomes. Of great interest to researchers is the question of the benefit and value PBL has to P-12 education, particularly related to non-cognitive outcomes addressing skills such as interpersonal skills, service to the community as well as students’ perceptions of the classroom experience. This literature views these outcomes as significant value-added aspects of the PBL experience. In this area, the results are so consistent that Sage (1996) suggests that the non-cognitive outcomes of PBL may be more critical than content acquisition. Other literature also shows strong support for the development of social skills such as supporting group work by developing a respect for others, helping students to learn from others, and listening to what others have to say (Cerezo, 2004; Goodnough & Cashion, 2006). Students in the PBL classes also reported that they are more motivated, self-efficacious, confident, enthusiastic, persistent, engaged, self-confident, willing to problem solve and share knowledge in PBL settings (Belland, Glazewski, & Ertmer, 2009; Brush & Saye, 2000; Cerezo, 2004; Tarhan, Ayar-Kayali, Urek, & Acar, 2008). Furthermore, some studies have found that students had positive perceptions of PBL, and believed that it improved the quality of the learning environment (Cerezo, 2004; Goodnough & Cashion, 2006; Gordon, Rogers, Comfort, Gavula, & McGee, 2001; Prettyman, Ward, Jauk, & Awad, 2012).

Finally, Gallagher and Gallagher (2013) demonstrated that PBL may be useful in identifying students with advanced academic potential who are typically overlooked when using standardized testing for identification procedures. In their study, findings suggested that using a well-designed, engaging curriculum such as PBL created a learning environment in which students emerged to show attributes of academic potential.

Students’ Perceptions of Classroom Quality

Students’ success and achievement have been tied to their perceptions about school and extend beyond what is measured by standards-based achievement tests. However, measurement of their perceptions has not been consistently assessed in part due to the unavailability of suitable instrumentation (Gable & Wolf, 1993; Haladyna & Thomas, 1979; Popham, 2001). Gentry and Owen (2004) developed the Student Perceptions of Classroom Quality (SPOCQ) scale in classrooms for gifted students, designed for use with secondary students to measure students’ perceptions of various aspects of classroom activities. The SPOCQ assesses their perceptions using five constructs that capture the quality of the classroom learning environment: (a) meaningfulness, (b) challenge, (c) choice, (d) self-efficacy, and (e) appeal. See Appendix B for a copy of the SPOCQ. Data from a large sample of secondary students indicated strong evidence for the internal consistency and validity of score interpretations of the instrument. The constructs measured by the SPOCQ constitute the substance of many curricular and instructional differentiation efforts, including initiatives targeted specifically toward the development of learning experiences for gifted students (Renzulli, Leppein, & Hays, 2000; Tomlinson, 1999). Considering previous studies found students in PBL reported increased self-efficacy, engagement, confidence, and an improvement in the learning environment, it seemed to fit well with PBL (Belland et al., 2009; Cerezo, 2004; Goodnough & Cashion, 2006; Gordon et al., 2001; Tarhan et al., 2008).

In light of the foregoing evidence in support of PBL, the purpose of this quasi-experimental study was to examine the effects of a PBL unit on pupils’ academic achievement and on their perceptions of the PBL environment in comparison with students learning the same content in teacher-centered, non-PBL classes. Specifically, the dependent variables were as follows: (a) students’ performance on standardized tests in science and (b) students’ perceptions of classroom quality according to the constructs of meaningfulness, challenge, choice, self-efficacy, and appeal as measured by the SPOCQ (Gentry & Owen, 2004) scale. This two-group, pre–post study was conducted in a gifted program for middle school students in a large suburban school district in the mid-Atlantic region of the United States, and sought to examine the effect of PBL on student achievement in a science unit and the students’ perceptions of classroom quality in comparison with a matched group of students receiving traditional teacher-centered instruction.

The current study examined two groups: a PBL group and a comparison group. The middle school level was chosen for this study because the school district had already identified the PBL unit in this study as equal to the text-based content of the traditional classrooms and was encouraging teachers to incorporate it into their instruction for gifted students. To create the PBL group, the teachers participated in a professional development program where the participants learned to teach using a middle school science unit developed according to the Stepien and Pyke (1997) five-phase PBL model. The PBL professional development session was 2 days long and was offered during the summer. The training was facilitated by a nationally recognized expert in PBL for P-12. During the professional development, the teachers experienced a simulation of PBL using one of the units and briefly reviewed the fundamentals of the Stepien and Pyke model. They learned how to follow the progression of a problem from beginning to end, ensuring a successful blend of student engagement, higher-order thinking, and required learning objectives. A PBL summer camp ran concurrently with the professional development. The camp provided teachers participating in the PBL workshop the opportunity to observe PBL in action from both the student and teacher perspectives. As a result of these sessions, each teacher developed a PBL coaching plan, which included a description of the class activities that would take place during their implementation of the instruction. This plan also included a list of skills targeted for development during the instruction of the unit, an outline of possible concepts and questions that might arise during instruction of the unit, a list of materials and resources that might be needed during instruction of the unit, and a list of assessment options during instruction of the unit. Three teachers who participated in this session and expressed their interest in implementing the unit were from one school. One of the teachers from this school attended the professional development session two consecutive summers. These three teachers were asked whether they would be willing to teach the unit and participate in data collection. They agreed. The principal at their school was approached to determine whether he was receptive to the teachers participating in data collection. The principal agreed. The teachers in the comparison group did not attend the professional development.

In the PBL group, three teachers from the same middle school building taught 223 students in the center program for highly able students a PBL unit, titled Ferret It Out, which was aligned with a subset of the objectives for the school district’s unit on the environment. In Ferret It Out, the students were the stakeholders as members of the Black Footed Ferret Recovery Implementation Team (BFFRIT). During the Problem Engagement of the Ferret It Out PBL unit, students discovered that they were tasked with identifying the different aspects of successful ferret reintroduction to prepare the newest test site in Ft. Collins, Colorado. During the Inquiry and Investigation phase of the unit, the students conducted research on self-selected topics based on the class-generated learning issues board developed following the students initial immersion in the problem during the problem engagement. During the Problem Definition phase, the students prioritized critical elements to ferret reintroduction, which potentially caused constraints and therefore needed to be considered in the problem resolution. The students then created a definition of the problem by responding to the problem using a structured format. For example, in the Ferret It Out unit a problem definition statement was, How can we improve chances of ferret survival while taking into account human safety and the rights of ranchers? During the Problem Resolution of the unit, the students worked in teams to develop their models for ferret reintroduction and created their presentation for sharing with the class. Also, during this fifth phase, the students reflected on their learning and the PBL experience.

The three PBL science teachers were all White females. Each had taught the target PBL unit at least one time previously. One of the teachers was in her second year of teaching, and the other two had 18 years of teaching experience. The teacher in her second year and one of the other teachers had taught the Investigations in Environmental Science course at the same school for the entire duration of their teaching careers. The other teacher with 18 years of experience was in her third year of experience teaching the Investigations in Environmental Science course at the school. Together, these teachers taught a total of 223 seventh-grade students in the center program for highly able students.

In the traditionally instructed comparison group, three teachers taught 252 students in the center program for highly able students in the district’s teacher-directed science curriculum for the same unit. In this comparison group, the teachers implemented the district’s traditional science curriculum for the Understanding Our Environment unit. This unit comprises a series of lab activities the students complete through lecture, reading, and worksheets. During the Understanding Our Environment unit, students make connections between Earth and Space Science to understand factors that determine the type of climate found in a particular geographic area, learn the characteristics of terrestrial and aquatic biomes, the plants and animals that live in them, and the adaptations of these plants and animals that allow them to survive in their environment. Students learn about watersheds, including the importance and vulnerability of the ecosystem of the Chesapeake Bay. Students set up a model aquatic ecosystem and observe the interactions between the living and non-living components over time. Students learn how the products of photosynthesis are used by producers and consumers, and explore the process of cellular respiration. Students gain an appreciation of the role of submerged aquatic vegetation and the effect of excess nutrients and sediments on them. Students explore the principles of sustainable development and apply what they have learned about environmentally friendly development to design a new community on an undeveloped parcel of land. Students visit a local waterway, and model the procedures and processes used by water quality monitors to collect data related to land use, water quality, habitat, and biological indicators to determine the relative health of the waterway.

Two female science teachers and one male science teacher taught the traditional, accelerated classrooms; all three teachers were White. The male teacher had 20 years of teaching experience, all teaching the Investigations in Environmental Science course, and had been teaching at the comparison school for the past 14 years. One of the female teachers had 12 years of teaching experience, all teaching this course, and had been teaching at the comparison school for 10 years. The other female teacher had 10 years of teaching experience, all teaching this course, and had been teaching at the comparison school for the past 9 years. A total of 252 of these teachers’ seventh-grade students in the center program for highly able students participated in the study. The two groups of student participants were purposefully matched on demographics as presented in Table 1.

Table

Table 1. Membership and Staffing of the Schools for the PBL and Comparison Groups.

Table 1. Membership and Staffing of the Schools for the PBL and Comparison Groups.

The PBL group was selected from a school that houses a program for highly able students. The school houses approximately 1,000 students in seventh and eighth grade. The comparison group was selected from a school, specifically because the school also houses a center program for highly able students and has similar demographics. The comparison school was also selected because, when asked, “What does instruction typically look like in your classroom?” all of the teachers at this school agreed that they stick closely to the district’s pacing guide and implement the district’s traditional science curriculum, which typically includes lecture and hands on lab activities. Based on the data the school system makes public, pass rates for the entire population of both schools are consistently above 95% on state reading tests and history tests, and above 75% on state mathematics tests.

All of the students in the study qualified for and participated in the center program for highly able students. Students are identified for the center program by a screening process utilizing a holistic multiple-criteria approach. To be found eligible, students must show evidence of a general intellectual ability requiring full-time educational services in an advanced setting. Sufficient academic documentation to support placement includes a review of exceptionally high ability test results, achievement test results, report cards, a rating scale of gifted behaviors, and student work samples. Optional documentation that may be considered in the screening file includes a parent questionnaire, awards and honors, and/or letters of recommendation.

Due to some data loss, there were totals of 449 complete data sets for the academic achievement measure and 443 for the SPOCQ Questionnaire. Data loss was due primarily to student absences on the day of testing for either the pre- or post-test, or the SPOCQ. Additional data loss was due to incorrectly completed or incomplete responses. These data were excluded from the study. Only complete data sets with correctly completed and complete responses were included in the data analysis. The total number of students participating in the study for the PBL group was 223. Of these 223 students, there were 206 complete pre/post-test data sets and 192 complete SPOCQ surveys. The total number of students participating in the study for the comparison group was 252. Of these 252 students, there were 243 complete pre/post-test data sets and 251 SPOCQ complete surveys.

The performance of the PBL group was compared with those of the students who received the traditional teacher-centered instruction on a standardized test developed using the district’s electronic test question repository, which mimics items on the state’s standardized for all students. To establish a baseline of prior knowledge, the students in each group took a 25-item pre-instruction assessment developed using the district’s repository of test items aligned to the state’s science content standards and the objectives of the instructional unit. At the end of the 3-week unit, the same assessment was administered again as a post-test, which allowed for study of the differences in student achievement. At the end of the unit, all students completed the 38-item SPOCQ to gain their assessment of the quality of instruction during instruction in the two settings.

Classroom observations of the teachers were conducted to check for the fidelity of implementation to the PBL model. A meta-cognitive coach checklist, designed by PBL researcher, Gallagher (n.d.), was used to rate the teachers’ actions and responses in the classroom on a 5-point Likert-type scale of 23 items. See Appendix A for a copy of the meta-cognitive coach checklist. The meta-cognitive coach checklist was designed to provide school administrators with an informal tool to check for fidelity of implementation. It had not been used previously in studies, and its psychometric properties are unknown. To ensure reliability of observation, two observers visited each teacher at the midpoint of the units and made summative judgments using the checklist to assess the degree to which the teachers practiced PBL with fidelity. For Teacher 1, the inter-rater reliability was 91% agreement, and both observers rated the teacher as fair. The overall rating of fair indicated the observers’ assessment of fidelity to the PBL model. For Teacher 2, the inter-rater reliability was 86% agreement, and both observers rated the teacher as fair. The overall rating of fair indicated the observers’ assessment of fidelity to the PBL model. For Teacher 3, the inter-rater reliability was 86%, and both observers rated the teacher as excellent. The overall rating of excellent indicated the observers’ assessment of fidelity to the PBL model.

Data were collected and analyzed using the Statistical Package for the Social Sciences (SPSS), and included the following: a t test for paired samples to compare the pre/post-test data for each of the groups, a t test for independent samples to compare the pre/post-test scores of the two groups, and SPOCQ data to compare the schools.

The pre-test data, as presented in Table 2, indicated no significant difference between the two groups of students (MPBL = 17.57, SD = 3.20; Mcomparison = 17.89, SD = 2.82), indicating that they had equal knowledge of the material prior to the implementation of the treatment. Levene’s Test for Equality of Variances indicated that the variability in the conditions of each of the groups did not differ significantly from each other, and that the equal variances were likely to have occurred by chance, F(1, 448) = .303, p = .582. The data from the post-tests revealed that each group increased in their knowledge of the content at a level that reached statistical significance (PBL = 23.50, SD = 1.40, p < .01; comparison = 22.54, SD = 2.06, p < .01).

Table

Table 2. Paired Sample t Test for PBL Group and Comparison Group Pre/Post-Test.

Table 2. Paired Sample t Test for PBL Group and Comparison Group Pre/Post-Test.

However, a comparison of the post-test scores of the two groups, as presented in Table 3, indicated that the PBL group outperformed the students taught in the traditional, didactic method at a level that reached significance (PBL post-test = 23.5, SD = 1.40; comparison post-test = 22.54, SD = 2.06, p < .01), thereby suggesting that PBL may be a better way to prepare students to perform well on state-like standard tests. Levene’s Test for Equality of Variances indicated that the variability in the conditions of each of the groups did differ significantly from each other and the variances were not likely to have occurred by chance, (F(1,488)=39.445, p<.001).

Table

Table 3. Independent-Samples t Test for PBL and Comparison Group Pre/Post-Test.

Table 3. Independent-Samples t Test for PBL and Comparison Group Pre/Post-Test.

As displayed in Table 4, with regard to the students’ perceptions of their classrooms as measured by the SPOCQ, an independent t test comparing the two groups’ results on the SPOCQ was conducted on the total score and each of the constructs. Levene’s Test for Equality of Variances indicated that the variability in the conditions for the challenge construct, F(1, 442) = .034, p = .853, the choice construct, F(1, 442) = .110, p = .740, the meaning construct, F(1, 442) = 1.085, p = .298, and the self-efficacy, F(1, 442) = .724, p = .395, construct did not differ significantly from each other, and that the equal variances were likely to have occurred by chance. Levene’s Test for Equality of Variances indicated that the variability in the conditions for the appeal construct was statistically significant, F(1, 442) = 9.152, p = .003.

Table

Table 4. Students’ Perceptions of Classroom Quality Means.

Table 4. Students’ Perceptions of Classroom Quality Means.

The difference in the comparison of means of the choice construct was statistically significant and favored the PBL group. The results for the challenge and meaning constructs were not statistically significant. The difference in the comparison of means on the appeal construct was statistically significant and favored the comparison group. The difference in the comparison of means on the self-efficacy construct was also statistically significant and favored the comparison group.

The results showed that on both the academic achievement measure and the SPOCQ, the scores generally favored the PBL group, thereby indicating that an indirect and constructivist approach to teaching can outperform a more direct approach to teaching, and that the students found the PBL setting as creating a better learning environment overall. This finding challenges the generally held belief that direct instruction is the most productive way to prepare gifted students for achievement on state-like standardized tests.

Particularly noteworthy are the differences on the SPOCQ. Looking at the five subscales, the difference in the comparison of means of the Choice subscale was statistically significant and favored the PBL group. This finding is not unexpected, as choice, often characterized as flexibility in the literature, is commonly found to support students’ positive perceptions of PBL (Chin & Chia, 2006; Lancaster et al., 1997; Nowak, 2001). This finding may also reflect the intentional design of the PBL curriculum to allow students to be self-directed, thus giving them choice over the direction of their learning.

The difference in the comparison of means on the Appeal subscale was statistically significant and favored the comparison group. PBL may not have the same sense of appeal, at least initially, to students as traditional instruction because of the active role that PBL demands of them as learners. The disorienting nature of PBL instruction appears to create an implementation dip or the inevitable difficulties people encounter when they are first learning new behaviors and beliefs (Fullan, 2001).

The difference in the means of the Self-Efficacy subscale was also statistically significant and favored the comparison group, as displayed in Table 5. Further analyses indicated strong positive correlations between self-efficacy and appeal, suggesting that when students have the belief in their ability to be successful, they found the learning more appealing. To test this relationship, a correlation between the appeal construct and the self-efficacy construct was conducted for the PBL group (r = .614, p < .01) and for the comparison group (r = .772, p < .01), as shown in Table 5.

Table

Table 5. Summary of Correlations on Appeal and Self-Efficacy Constructs for PBL and Comparison Group.

Table 5. Summary of Correlations on Appeal and Self-Efficacy Constructs for PBL and Comparison Group.

These results appear to suggest that the PBL environment was new for the students and the uncertainty of learning in a new format may have negatively affected their sense of efficacy and therefore how appealing the learning environment was. Previous literature provides support for this finding. For example, Diggs (1997) provided strong evidence that students’ attitudes might be attributed to the design and delivery of instruction.

The finding indicating that the Challenge subscale was not statistically significant may reflect the rigorous curriculum of the gifted program and suggests that students in this school district perceive themselves to be challenged no matter what environment they are in. Likewise, the lack of significance on the Meaningfulness subscale may reflect the nature of gifted students to find meaning in their learning regardless of the context.

Several conclusions can be drawn from this study. First, this study demonstrated that focused professional development on learning to teach PBL can lead teachers to implement the model with fidelity. That is, the treatment teachers taught the unit as it was expected to be taught. Second, the students’ results indicated that an indirect and constructivist approach to teaching used with the PBL treatment group can outperform a more direct approach to teaching in scores on academic achievement measures. Furthermore, the students found the PBL setting to create more choice in their learning. This finding challenges the generally held belief that direct instruction is the most productive way to prepare students, in particular gifted students, for achievement on state-like standardized tests (Au, 2007).

Third, the results of the SPOCQ on the subscale of Choice favoring the PBL group suggest that choice is an attribute of learning that students value, and it is not unexpected. Choice, often characterized as flexibility, has been found to support students’ positive perceptions of PBL (Chin & Chia, 2006; Lancaster et al., 1997; Nowak, 2001). This finding further asserts the comprehensive benefit of PBL in both cognitive and non-cognitive domains as not only an effective but also a preferable option for instruction. Conversely, the finding indicating that the Challenge subscale was not statistically significant may reflect the rigorous curriculum of the gifted program and also that the students in this school district perceive themselves to be challenged regardless of the environment, and needs further investigation. Likewise, the lack of significance on the Meaning subscale may reflect the nature of gifted students to find meaning in their learning regardless of the context, and again, more research is needed to understand this relationship.

Furthermore, PBL may not have the same sense of appeal, at least initially, to students as traditional instruction because of the active role that PBL demands of them as learners. The non-traditional and perhaps disorienting nature of PBL instruction appears to create in these students what Fullan (2001) called “an implementation dip,” or the inevitable difficulties people encounter when they experience a change, in this case, in classroom instruction. Further analyses indicate strong and positive correlations between self-efficacy and appeal, suggesting that when students have belief in their ability to be successful, they find it more appealing. The positive correlation between the appeal construct and the self-efficacy construct suggests that the PBL environment was new for these students, and that the uncertainty of learning in a new format may have negatively affected their sense of efficacy, and therefore how appealing the learning environment was to them. Previous research by Diggs (1997) provided compelling evidence that students’ attitudes might be attributed to the design and delivery of instruction. There is still a great deal to know about this in other subject matter areas.

Limitations and Strengths

Often in studies of teacher education, fidelity of implementation is a barrier to good data. That is, many teachers fail to adopt the principles of the new strategy in ways amenable to research. That was not the case in this study. In all ways, the professional development, the implementation of the PBL model, and the data gathering were not compromised. As such, none of the usual limitations of classroom-based research are present. Furthermore, studies of teaching are limited by small samples and the lack of comparison groups. Neither of those limitations was a constraint in this case. The study involved a large number of students who participated, and the populations were very equally matched.

However, the results of the study are specific to the population studied, namely, middle school gifted students. In addition, these results are specific to the district examined in the study and to the content of the Ferret It Out unit, but all of these limitations are common contexts that limit many informative studies in schools. As this study examined a specific instructional model, it was necessary to collect data from classrooms where the instructional model was being used; therefore, the study was further limited by the fact that the teachers were not selected randomly, but rather volunteered to be a part of the study after having been offered the opportunity to participate.

In addition, these results align with school improvement priorities and goals. For a number of years, district staff had been involved in writing, pilot-testing, and revising PBL units using a collaborative expert-practitioner model of curriculum development. However, while there was substantial anecdotal evidence that the units were engaging and appropriate, a more widespread adoption of these units for implementation had not been embraced. This was in part due to teachers’ concerns regarding content acquisition and adequate preparation for standardized tests that cannot be summarily dismissed.

Finally, the findings related to the meta-cognitive coach checklist provided implications for building strong working relationships among teachers. Facilitating instruction with the expectations of the meta-cognitive coach checklist is a transition for many teachers. The checklist can be used as a tool for teachers to engage in dialogue about the observation of behaviors specific to the effective implementation of PBL instruction. Because the tool is not inherently evaluative, the process of engaging in this dialogue has the potential to support teachers in their transition by serving as a structure for building strong working relationships through feedback and reflection. This study also demonstrated that PBL can and does challenge the prevailing assumption that the best way to teach in a standardized test environment is to use traditional, teacher-directed instruction. Therefore, one practical application of this study is for PBL to be embedded in the continuing professional development of educators both in advanced degree programs for teachers and in local teacher professional development sessions.

This is consistent with a recent report from the Association for Career and Technical Education, National Association of State Directors of Career Technical Education Consortium and Partnership for 21st-Century Skills (Up to the Challenge, 2010), which highlights the need for skills such as communication, effectively defining problems and developing solutions, motivation and persistence, and the ability to work with others in team settings, in a creative, innovative society concerned with preparing students for college and career readiness. Therefore, scholars have become concerned with non-cognitive outcomes of classroom instruction, not only in terms of these skills but also in terms of the quality and meaningfulness of the classroom experience (Gentry & Owen, 2004). PBL is intentionally designed to align with content standards. This is done through the use of the ill-structured problem, which leads students to the content required in the core curriculum. Consequently, students’ questions become the basis of instruction, empowering the students with a sense of inquiry. As the core content emerges naturally through the problem, instruction can focus on bringing greater complexity to their understanding not only of content but also by adding depth and breadth through developing self-directed learners as well as those considered important for success in the 21st century. Accordingly, it is critical students, in particular gifted students, experience PBL.

As the 21st century progresses, the need for higher-order thinking skills, self-regulated learning habits, and problem-solving skills have emerged as learning expectations. However, the adoption of any instructional innovation, including PBL, in public education is a complex process with constraints and influences at many levels. Considering that students can now access considerable amounts of information in ways that were not possible even as recently as a decade ago and the increasing complexity of a global society naturally providing an abundance of suitable ill-structured real-world problems, PBL can allow teachers to create more engaging learning environments while meeting accountability expectations.

Meta-Cognitive Coach Checklist

Rate each item on a scale of 1 to 5, with 1 being strongly agree, 5 being strongly disagree.

Behaviors

Table

Table

Reasoning process

Table

Table

Independent study

Table

Table

Overall process

Table

Table

General comments

 What did the teacher do that was most useful for learning?

 What suggestions for improvement would you make?

 What components of PBL did the teacher do well?


                        figure

Student Perceptions of Classroom Quality (SPOCQ).

We would like to thank Shelagh Gallagher for suggestions on earlier drafts of this article.

Authors’ Note
The authors are responsible for all content in the article.

Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.

Abrams, L. M., Pedulla, J. J., Madaus, G. F. (2003). Views from the classroom: Teachers’ opinions of statewide testing programs. Theory Into Practice, 42, 18-32. doi:10.1207/s15430421tip4201_4
Google Scholar | Crossref
Au, W. (2007). High-stakes testing and curricular control: A qualitative metasynthesis. Educational Researcher, 36, 258-267. doi:10.3102/0013189X07306523
Google Scholar | SAGE Journals
Barrows, H. S. (1988). The tutorial process. Springfield: Southern Illinois University School of Medicine.
Google Scholar
Barrows, H. S. (1994). Practice-based learning. Springfield: Southern Illinois University School of Medicine.
Google Scholar
Barrows, H. S., Tamblyn, R. M. (1980). Problem-based learning: An approach to medical education. New York, NY: Springer.
Google Scholar
Belland, B. R., Glazewski, K. D., Ertmer, P. A. (2009). Inclusion and problem-based learning: Roles of students in a mixed-ability group. RMLE Online: Research in Middle Level Education, 32, 1-19. doi:10.1080/19404476.2009.11462062
Google Scholar | Crossref
Brown, E., Avery, L., VanTassel-Baska, J., Worley, B. B., Stambaugh, T. (2006). A five-state analysis of gifted education policies. Roeper Review, 29, 11-23. doi:10.1080/02783190609554379
Google Scholar | Crossref
Bruner, J. (1962). The act of discovery. In Bruner, J. S. (Ed.), On knowing: Essays for the left hand (pp. 81-96). Cambridge, MA: Harvard University Press.
Google Scholar
Brush, T., Saye, J. (2000). Design, implementation, and evaluation of student-centered learning: A case study. Educational Technology Research & Development, 48, 79-100. doi:10.1007/BF02319859
Google Scholar | Crossref
Cerezo, N. (2004). Problem-based learning in the middle school: A research case study of the perceptions of at-risk females. RMLE Online: Research in Middle Level Education, 27, 1-13.
Google Scholar
Chin, C., Chia, L. (2004). Problem-based learning: Using students’ questions to drive knowledge construction. Science Education, 88, 707-727. doi:10.1002/sce.10144
Google Scholar | Crossref
Chin, C., Chia, L. (2006). Problem-based learning: Using ill-structured problems in biology project work. Science Education, 90, 44-67. doi:10.1002/sce.20097
Google Scholar | Crossref
Diggs, L. L. (1997). Student attitude toward and achievement in science in a Problem-Based Learning educational experience (Doctoral dissertation). Retrieved from Dissertations & Theses. (Publication No. AAT 9842589)
Google Scholar
Dods, R. F. (1997). An action research study of the effectiveness of problem-based learning in promoting the acquisition and retention of knowledge. Journal for the Education of the Gifted, 20, 423-437. doi:10.1177/016235329702000406
Google Scholar | SAGE Journals
Ertmer, P., Simons, K. (2006). Jumping the PBL implementation hurdle: Supporting the efforts of K-12 teachers. The Interdisciplinary Journal of Problem-Based Learning, 1, 40-54. doi:10.7771/1541-5015.1005
Google Scholar | Crossref
Fullan, M. (2001). Leading in culture of change. San Francisco, CA: Jossey-Bass.
Google Scholar
Gable, R. K., Wolf, M. B. (1993). Instrument development in the affective domain. Boston, MA: Kluwer.
Google Scholar | Crossref
Gallagher, S. A. (n.d.). Meta-cognitive Coach Checklist [Measurement Instrument]. Unpublished instrument.
Google Scholar
Gallagher, S. A. (2001). But does it work? Testing the efficacy of problem-based learning: A review of the literature and research agenda for educators of the gifted. In Colangelo, N., Wallace, S. G. (Eds.), Talent development: Proceedings from the 1998 Henry B. and Jocelyn Wallace National Research Symposium (pp. 179-204). Scottsdale, AZ: Great Potential Press.
Google Scholar
Gallagher, S. A. (2009). Designed to fit: Educational implications of gifted adolescents’ cognitive development. In Dixon, F. (Ed.), Programs and services for gifted secondary students (pp. 3-20). Waco, TX: Prufrock Press.
Google Scholar
Gallagher, S. A. (2012). Building bridges: Using research from the Big Five, MBTI, overexcitabilites and Perry to explore personality differences of gifted youth. In Neville, C. S., Piechowski, M. M., Tolan, S. S. (Eds.), Off the charts: Asynchrony and the gifted child (pp. 56-118). Unionville, NY: Royal Fireworks Press.
Google Scholar
Gallagher, S. A., Gallagher, J. J. (2013). Using problem-based learning to explore unseen academic potential. Interdisciplinary Journal of Problem-Based Learning, 7, 111-131. doi:10.7771/1541-5015.1322
Google Scholar | Crossref
Gallagher, S. A., Stepien, W. J. (1996). Depth versus breadth in problem-based learning: Content acquisition in American studies. Journal for the Education of the Gifted, 19, 257-275. doi:10.1177/016235329601900302
Google Scholar | SAGE Journals
Gallagher, S. A., Stepien, W. J., Rosenthal, H. (1992). The effects of problem-based learning on problem solving. Gifted Child Quarterly, 36, 195-200. doi:10.1177/001698629203600405.
Google Scholar | SAGE Journals
Gentry, M., Owen, S. V. (2004). Secondary student perceptions of classroom quality: Instrumentation and differences between advanced/honors and nonhonors classes. Journal of Secondary Gifted Education, 16, 20-29. doi:10.4219/jsge-2004-464
Google Scholar | SAGE Journals
Goodnough, K., Cashion, M. (2006). Exploring problem-based learning in the context of high school science: Design and implementation Issues. School Science and Mathematics, 106, 280-300. doi:10.1111/j.1949-8594.2006.tb17919.x.
Google Scholar | Crossref
Gordon, P. R., Rogers, A. M., Comfort, M., Gavula, N., McGee, B. P. (2001). A taste of problem-based learning increases achievement of urban minority middle-school students. Educational Horizons, 79, 171-175. Retrieved from http://www.jstor.org/stable/42927061
Google Scholar
Haladyna, T., Thomas, G. (1979). The attitudes of elementary school children toward school and subject matters. Journal of Experimental Education, 48, 18-23. doi:10.1080/00220973.1979.11011707
Google Scholar | Crossref
Hitchcock, M. A., Mylona, Z. E. (2000). Teaching faculty to conduct problem-based learning. Teaching and Learning in Medicine, 12, 52-57. doi:10.1207/S15328015TLM1201_8
Google Scholar | Crossref | Medline
Hmelo, C. E., Holton, D. L., Allen, J. K., Kolodner, J. L. (1997). Designing for understanding: Children’s models of lungs. In Shafto, M. G., Langley, P. (Eds.), Proceedings of the nineteenth annual meeting of the Cognitive Science Society (pp. 313-318). Mahwah, NJ: Lawrence Erlbaum.
Google Scholar
Hockett, J. A. (2009). Curriculum for highly able learners that conforms to general education and gifted education quality indicators. Journal for the Education of the Gifted, 32, 394-440.
Google Scholar | SAGE Journals
Jonas, H., Etzel, S., Barzansky, B. (1989). Undergraduate medical education. The Journal of the American Medical Association, 262, 1011-1019. doi:10.1001/jama.1989.03430080031005
Google Scholar | Crossref
Kang, N. H., DeChenne, S. E., Smith, G. (2012). Inquiry learning of high school students through a problem-based environmental health science curriculum. School Science and Mathematics, 112, 147-158. doi:10.1111/j.1949-8594.2011.00128.x
Google Scholar | Crossref
Lancaster, C., Bradley, E., Smith, I. K., Chessman, A., Stroup-Benham, C. A., Camp, M. (1997). The effect of PBL on students’ perceptions of learning environment. Academic Medicine, 72, S10-S12. doi:10.1097/00001888-199710001-00004
Google Scholar | Crossref | Medline
Lehman, J. D., George, M., Buchanan, P., Rush, M. (2006). Preparing teachers to use problem-centered, inquiry-based science: Lessons from a four-year professional development project. The Interdisciplinary Journal of Problem-Based Learning, 1, 76-99. doi:10.1097/00001888-199710001-00005
Google Scholar | Crossref
Maker, C. J., Nielson, A. B. (1995). Teaching models in education of the gifted (2nd ed.). Austin, TX: Pro-Ed.
Google Scholar
Maker, C. J., Schiever, S. (1991). Enrichment and acceleration: An overview and new directions. In Davis, G., Colangelo, N. (Eds.), Handbook of gifted education (pp. 99-110). Boston, MA: Allyn & Bacon.
Google Scholar
Moon, T. R., Brighton, C. B., Callahan, C. M. (2003). State standardized testing programs: Friend or foe of gifted education? Roeper Review, 25, 49-60. doi:10.1080/02783190309554199
Google Scholar | Crossref
Nowak, J. A. (2001). The implications and outcomes of using problem-based learning to teach middle school science (Doctoral dissertation). Retrieved from Dissertations & Theses. (Publication No. AAT 3024268)
Google Scholar
Pedersen, S., Liu, M. (2003). The transfer of problem-solving skills from a problem-based learning environment: The effect of modeling an expert’s cognitive processes. Journal of Research on Technology in Education, 35, 303-320. doi:10.1080/15391523.2002.10782388
Google Scholar | Crossref
Popham, W. J. (2001). The truth about testing: An educator’s call to action. Alexandria, VA: Association for Supervision and Curriculum Development.
Google Scholar
Prettyman, S. S., Ward, C. L., Jauk, D., Awad, G. (2012). 21st century learners: Voices of students in a One-to-One STEM Environment. Journal of Applied Learning Technology, 2, 6-15.
Google Scholar
Reis, S. M., Westberg, K. L., Kulikowich, J., Caillard, F., Hébert, T., Plucker, J., . . . Smist, J. M. (1993). Why not let high ability students start school in January? The Curriculum Compacting Study (Research Monograph 93106). Storrs, CT: NRC/GT.
Google Scholar
Renzulli, J. S., Leppein, J., Hays, T. (2000). The multiple menu model: A practical guide for developing differentiated curriculum. Mansfield Center, CT: Creative Learning Press.
Google Scholar
Sage, S. M. (1996, April). A qualitative examination of problem-based learning at the K-8 level: Preliminary findings. Paper presented at the annual meeting of the American Educational Research Association. Retrieved from http://mutex.gmu.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&;db=eric&AN=ED398263&site=ehost-live
Google Scholar
Savery, J. R. (2006). Overview of problem-based learning: Definitions and distinctions. Interdisciplinary Journal of Problem-Based Learning, 1, 9-20. doi:10.7771/1541-5015.1002
Google Scholar | Crossref
Savery, J. R., Duffy, T. M. (1995). Problem-based learning: An instructional model and its constructivist framework. Educational Technology, 35, 31-38.
Google Scholar
Siegle, D., Wilson, H., Little, C. (2013). A sample of gifted and talented educators’ attitudes about academic acceleration. Journal of Advanced Academics, 24, 27-51. doi:10.1177/1932202x12472491
Google Scholar | SAGE Journals
Stepien, W. J., Pyke, S. (1997). Designing problem-based learning units. Journal for the Education of the Gifted, 20, 380-400. doi:10.1177/016235329702000404
Google Scholar | SAGE Journals
Strobel, J., van Barneveld, A. (2009). When is PBL more effective? A Meta-synthesis of Meta-analyses comparing PBL to conventional classrooms. The Interdisciplinary Jouarnal of Problem-Based Learning, 3 (1) 44-58. doi: http://dx.doi.org/10.7771/1541-5015.1046
Google Scholar
Sungur, S., Tekkaya, C., Geban, O. (2006). Improving achievement through problem-based learning. Journal of Biological Education, 40, 155-160. doi:10.1080/00219266.2006.9656037
Google Scholar | Crossref
Swan, K., Vahey, P., van, ‘t, Hooft, M., Kratcoski, A., Rafanan, K., Stanford, T., . . . Cook, D. (2013). Problem-based learning across the curriculum: Exploring the efficacy of a cross-curricular application of preparation for future learning. Interdisciplinary Journal of Problem-Based Learning, 7, 91-110. doi:10.7771/1541-5015.1307
Google Scholar | Crossref
Tarhan, L., Ayar-Kayali, H., Urek, R., Acar, B. (2008). Problem-based learning in 9th grade chemistry class: “Intermolecular forces.” Research in Science Education, 38, 285-300. doi:10.1007/s11165-007-9050-0
Google Scholar | Crossref
Tomlinson, C. A. (1999). How to differentiate instruction in mixed ability classrooms. Alexandria, VA: Association for Supervision and Curriculum Development.
Google Scholar
Up to the Challenge . (2010). The role of career and technical education and 21st Century Skills in College and Career Readiness. Retrieved from http://www.p21.org/storage/documents/CTE_Oct2010.pdf
Google Scholar
VanTassel-Baska, J., Brown, E. (2007). Towards best practice: An analysis of the efficacy of curriculum models in gifted education. Gifted Child Quarterly, 51, 342-358. doi:10.1177/0016986207306323
Google Scholar | SAGE Journals
VanTassel-Baska, J., Stambaugh, T. (2006). Project Athena: A pathway to advanced literacy development for children of poverty. Gifted Child Today, 29, 58-63. doi:10.1177/0016986207306323
Google Scholar | SAGE Journals
Wetzel, M. S. (1996). Techniques in medical education: Problem-based learning. Postgraduate Medical Journal, 72, 474-477. doi:10.1136/pgmj.72.850.474
Google Scholar | Crossref | Medline
Wirkala, C., Kuhn, D. (2011). Problem-based learning in K-12 education: Is it effective and how does it achieve its effects? American Educational Research Journal, 48, 1157-1186. doi:10.3102/0002831211419491
Google Scholar | SAGE Journals | ISI
Zhang, M., Parker, J., Eberhardt, J., Passalacqua, S. (2011). “What’s so terrible about swallowing an apple seed?” Problem-based learning in kindergarten. Journal of Science Education and Technology, 20, 468-481. doi:10.1007/s10956-011-9309-0
Google Scholar | Crossref
Zsiga, P. L., Webster, M. (2007). Why should secondary educators be interested in self-directed learning? International Journal of Self-Directed Learning, 4, 58-68.
Google Scholar

About the Authors

Anne K. Horak is an assistant research professor in the College of Education and Human Development at George Mason University, Fairfax, Virginia. She serves as the Coordinator for Project ExCEL, a federally funded Jacob K. Javits grant. Project ExCEL uses Problem-Based Learning as a platform to find and serve high ability low income students. Prior to joining the faculty at Mason, she served as Advanced Academic Program Specialist for Fairfax County Public Schools. Her research interests include Problem-Based Learning, Gifted Education, Educational Policy and Teacher Professional Development.

Gary R. Galluzzo is a professor emeritus in the College of Education and Human Development at George Mason University, Fairfax, Virginia. His research interests include investigations into how students become teachers, curriculum reform in teacher education, program evaluation in teacher education, education reform, and preparing teachers to be the agents of school change.