Knowledge and Perceptions of Competency-Based Medical Education in Diagnostic Radiology Post-Graduate Medical Education: Identifying Priorities and Developing a Framework for Professional Development Activities

Introduction: We evaluated knowledge and perceptions of an established Competency-Based Medical Education (CBME) model developed by the Royal College of Physicians and Surgeons of Canada, Competence by Design (CBD), and identified evidence-informed priorities for professional development activities (PDAs). Materials and Methods: Teaching faculty and residents at a single, large diagnostic radiology post-graduate medical education (PGME) program were eligible to participate in this cross-sectional, survey-based study. Knowledge of CBD was evaluated through multiple choice questions (MCQs), which assessed participants’ understanding of major principles and terms associated with CBD. Participants’ perceptions of the anticipated impact of CBD on resident education and patient care were evaluated and priorities for PDAs were identified, which informed a framework for CBD PDAs. Results: Fifty faculty and residents participated. The faculty and resident response rates were 11.6% (n = 29/249) and 55.3% (n = 21/38), respectively. The mean ± standard deviation overall score on MCQs was 39.0% ± 20.4%. The majority of participants perceived the impact of CBD on resident education to be equivocal and to not impact patient care. Knowledge of CBD was not statistically significantly associated with participants’ perceptions of the impact of CBD on either resident education or patient care (P > .05). Delivery of high-quality feedback was the greatest priority identified for PDAs. Discussion: Our results and proposed CBD PDAs framework may help to guide diagnostic radiology PGME programs in designing evidence-informed PDAs, which may meaningfully contribute to the successful implementation of CBD in diagnostic radiology PGME. As diagnostic radiology PGME programs throughout the world increasingly implement CBME models, evidence-informed PDAs will become of increasing importance. Graphical Abstract


Introduction
Competency-Based Medical Education (CBME) is being increasingly implemented in post-graduate medical education (PGME) programs, including diagnostic radiology. 1 Among key components of CBME is the application of a framework of competencies, which are outcomes-based and are organized to delineate progressive achievement of competence. 2,3 Workplace-based assessments, which foster increased opportunities for feedback and coaching, formulate the foundation of assessment in CBME. 4 Several benefits of CBME in PGME have been described in the literature, including the delineation of standardized specialty-specific competencies, 5,6 increased opportunities for the provision of feedback and coaching, 6,7 facilitation of modified teaching and learning opportunities for individual residents, 5,6 and early identification of residents who may benefit from additional teaching and learning supports. 5 Despite the benefits of CBME, there remains limited knowledge as to how to effectively implement CBME models in diagnostic radiology PGME training.
One of the most established CBME models is Competence by Design (CBD), which was developed by the Royal College of Physicians and Surgeons of Canada (RCPSC). 8 Competence by Design integrates CBME through the application of a framework of competencies with traditional time-based PGME training. [8][9][10] Since 2017, the RCPSC has progressively transitioned the PGME training of residents in various specialties in Canada to CBD. 11 Previously, the PGME training of residents in diagnostic radiology at Queen's University, Kingston, Ontario was transitioned to CBD as part of an institutional initiative in 2017. 12,13 Recently, the PGME training of residents in diagnostic radiology in Canada nationally transitioned to CBD in July 2022 for the incoming resident cohort, and thus diagnostic radiology PGME programs nationally are actively preparing. 9,14,15 Implementation of CBME models, such as CBD, requires active involvement of multiple stakeholders not only to ensure successful implementation but also to assure that the intended educational outcomes of CBME are manifested. 6,16 In other specialties, provision of professional development activities (PDAs) pertaining to CBD has been identified as being among those strategies integral to achieving these goals. 6,[17][18][19][20][21][22] The objectives of this study were to evaluate diagnostic radiology faculty and residents' knowledge and perceptions of CBD, an established CBME model, and to identify evidence-informed priorities for associated PDAs. Based on our observations and review of the literature, we subsequently propose a framework for PDAs, which may be used by diagnostic radiology PGME programs in preparation for the implementation of CBD or other CBME models. Such frameworks will become of increasing importance as diagnostic radiology and radiology subspecialty PGME programs throughout the world increasingly implement CBME models.

Methods
This was a cross-sectional, survey-based study. Approval was received from the institutional review board (REB H21-02493). Diagnostic radiology teaching faculty and residents at a single, large diagnostic radiology PGME program were eligible to participate. Data collection was by means of the Qualtrics survey tool (Provo, Utah). 23 An anonymized link to the survey was electronically distributed to eligible participants through the department's electronic mailing list. Eligible participants were provided the opportunity to complete the survey during a 6-week period. The survey was divided into 4 sections: (1) academic demographics, (2) knowledge of CBD, (3) perceptions of CBD, and (4) identification of priorities for PDAs.

Academic Demographics
Academic demographic information was collected, including academic level, academic rank, years in practice, and area of specialization.

Knowledge of CBD
Knowledge of CBD was evaluated through multiple choice questions (MCQs) (Table S1) adapted from Stefan et al (2019), 22 which assessed understanding of major principles and terms utilized in CBD. For each MCQ, participants were asked to select the single correct response out of 5 possible responses. The number of MCQs administered was chosen to balance the evaluation of depth of knowledge with survey length. 24 An overall score attained by each participant on these MCQs was calculated, which served as an indicator of knowledge of CBD.

Perceptions of CBD
Participants' perceptions of the anticipated impact of CBD on resident education and patient care were evaluated through comparison with the current time-based PGME training paradigm. Participants were asked whether they perceived that CBD would provide better educational experiences than the current time-based PGME training paradigm and were asked to rank their response on a 5-point Likert scale. Participants were also asked whether they perceived that CBD would impact patient care in the radiology department as compared to the current time-based PGME training paradigm and were asked to select a single response from among 5 categorical responses: "yes, patient care will be negatively affected", "patient care may or may not be negatively affected", "no, patient care will not be affected", "patient care may or may not be positively affected", or "yes, patient care will be positively affected".

Professional Development Activities
Finally, participants were asked to select priority topics for PDAs from among the following topics previously identified in the literature in other specialties: overview of general concepts of CBD, delivery of high-quality feedback, receiving feedback on teaching, completion of learner assessments following a clinical encounter, and principles of resident promotion and advancement. 22 Participants were also able to provide additional priority topics for PDAs through the provision of free text responses.

Statistical Analyses
Data were summarized by descriptive statistics. Bivariate analyses were conducted as appropriate. Where data points were not provided by participants, the proportion of participants was calculated using the number of participants for which data was available as the denominator.

Participants
Fifty faculty and residents participated in this study. The response rate for faculty was 11.6% (n = 29/249) and that for residents was 55.3% (n = 21/38). Most residents (52.4%) who participated were in their second or third year of PGME training. The academic rank of the majority of faculty who participated was either that of clinical assistant professor (27.6%) or clinical associate professor (24.1%). Participants' academic demographics are summarized in Table 1.

Perceptions of CBD
Participants' perceptions of the anticipated impact of CBD on resident education are summarized in Figure 1. Overall, 50.0% of participants perceived that CBD may or may not provide better educational experiences for residents than the current time-based PGME training paradigm. 34.0% of participants perceived that CBD would probably provide better educational experiences for residents while 14.0% of participants perceived that CBD would probably not provide better educational experiences for residents than the current time-based PGME training paradigm. No participants perceived that CBD would not definitely or would definitely provide better educational experiences for residents than the current time-based PGME training paradigm. Likewise, both faculty (44.8%) and residents (57.1%) most often perceived that CBD may or may not provide better educational experiences for residents than the current time-based PGME training paradigm. Perceptions of the anticipated impact of CBD on residents' educational experiences as compared to the current time-based PGME training paradigm were not statistically significantly variable between faculty and residents (P > .05) though faculty (20.7%) more often than residents (4.8%) tended to perceive the impact of CBD to be detrimental to the educational experiences of residents as compared to the current time-based PGME training paradigm.
Participants' perceptions of the anticipated impact of CBD on patient care are summarized in Figure 2. Only one participant (2.0%) perceived that the implementation of CBD would positively impact patient care in the radiology department as compared to the current time-based PGME training paradigm. Overall, 40.0% of participants perceived that the implementation of CBD would not impact patient care in the radiology department as compared to the current time-based PGME training paradigm. Perceptions of the anticipated impact of CBD on patient care in the radiology department as compared to the current time-based PGME training paradigm were significantly variable between faculty and residents (P <  .05), with faculty (34.5%) most often perceiving that the implementation of CBD may or may not positively impact patient care and residents (57.1%) most often perceiving that the implementation of CBD would not impact patient care in the radiology department as compared to the current timebased PGME training paradigm. Participants' overall performance on MCQs to evaluate CBD knowledge was not observed to be statistically significantly associated with participants' perceptions of the impact of CBD on either resident education or patient care in the radiology department (P > .05).

Priorities for PDAs
Participants identified information on the delivery of high-quality feedback (52.0%), principles of resident promotion and advancement (50.0%), and general concepts of CBD (50.0%) as priorities for PDAs pertaining to CBD (Figure 3). Identified priorities for PDAs pertaining to CBD were not statistically significantly variable between faculty and residents (P > .05).

Discussion
Despite the increasing emphasis on CBME in PGME, little information remains available to guide diagnostic radiology PGME programs in the implementation of CBME models. Literature from other specialties may be limitedly applicable to diagnostic radiology PGME given the unique nature of diagnostic radiology PGME training thus further exacerbating the impact of the paucity of literature on the implementation of CBME models, such as CBD. This is the first study which identified evidence-informed priorities for associated PDAs in diagnostic radiology PGME.
Overall, our observations suggest that knowledge of CBD is poor. At the time of survey distribution, only a few PDAs, such as grand rounds presentations pertaining to CBD, had been previously provided at our institution. These observations are comparable to that of Stefan et al (2019) which utilized MCQs from which those in our study were adapted to evaluate knowledge of CBD among emergency medicine faculty and residents. 22 In contrast to our observations of diagnostic radiology faculty's knowledge of CBD being comparable to that of residents, Bogie et al (2021) observed the self-reported overall knowledge of CBD of psychiatry faculty to be greater than that of residents. 25 Bogie et al (2021) also observed psychiatry residents' self-reported overall knowledge of CBD to statistically significantly improve following a targeted PDA, demonstrating the positive impact of targeted PDAs on knowledge of CBD. 25 Despite the growing body of literature demonstrating the benefits of CBME in PGME, 5-7 our study found that diagnostic radiology faculty and residents are not aware of the advantages of CBD. Most participants in our study perceived the anticipated impact of CBD on residents' educational experiences as compared to the current time-based PGME training paradigm to be equivocal, which may suggest limited confidence that the implementation of CBD will manifest intended educational outcomes. The RCPSC Pulse Check 6 describes 2 measures pertaining to the evaluation of the implementation of CBD: fidelity and integrity. Fidelity is described as the extent to which "key elements" of CBD have been implemented while integrity is described as "[embracement] of the underlying principles [of CBD] and [appreciation of the rationale] for change". 6 Our observations suggest that PDAs be targeted to assure the integrity of the implementation of CBD in diagnostic radiology PGME. We recommend that diagnostic radiology PGME programs planning to implement CBD emphasize the educational advantages of CBME over the current time-based PGME training paradigm and include associated evidence where applicable.
The assertion that improved educational experiences of residents shall translate to improved patient care underlies the rationale for the development of CBME. 5,26 However, most participants in our study likewise did not perceive the anticipated impact of CBD to affect patient care in the radiology department. Failure of stakeholders to perceive meaningful impacts of CBD poses a significant challenge to its successful implementation. Poor "buy in" to CBD has been recurrently cited in the medical education literature across multiple specialties. 6,15,27 While our study did not demonstrate that knowledge of CBD was associated with perceptions of CBD, whether poor knowledge of CBD is contributory to perceptions of CBD must nonetheless be considered. 22 Investment in acquiring diagnostic radiology-specific data to establish an evidence-base to demonstrate the benefits of CBD to all stakeholders shall be integral to the successful implementation of CBD in diagnostic radiology PGME. 7,15 Identification of all stakeholders in CBD implementation in each PGME program is necessary as involvement of all stakeholders in PDAs pertaining to CBD is essential. Though participants in our study were solely limited to faculty and residents, we have identified learners, core academic faculty, community faculty, peer assessors, including senior residents and fellows, competence committee members, academic advisors, allied health professionals, and PGME program administrators as stakeholders in the implementation of CBD in diagnostic radiology PGME who ought to be involved in PDAs pertaining to CBD (Table 2).
Various domains of faculty development have been previously described. However, there remains a paucity of literature documenting the use of these domains in diagnostic radiology PGME. Steinert (2014) previously described 5 domains of faculty development in the health professions: "teaching improvement", "leadership and management", "research capacity building", "academic and career development", and "organizational change". 28 Applying Steinert's (2014) model of faculty development, 28 Lupi et al (2018) delineated the requisite skills and knowledge required of identified stakeholders in the implementation of Core Entrustable Professional Activities (EPAs) for Entrance into Residency developed by the Association of American Medical Colleges. 29 There is considerable overlap between these skills and knowledge that were identified as priorities for PDAs in our study.
To inform PDAs, we suggest application of Steinert's 5 domains of faculty development 28 in keeping with the approach utilized by Lupi et al (2018) 29 to delineate the requisite skills required of each identified stakeholder in the implementation of CBD in diagnostic radiology PGME. Suggested requisite skills required of each identified stakeholder in CBD implementation in diagnostic radiology PGME are summarized in Table 2. Subsequent mapping of our identified priorities for PDAs pertaining to CBD to these aforementioned domains revealed their consistent delineation to the teaching improvement domain ( Table 2). Although it is important for all of these domains to be addressed, we recommend PGME programs implementing CBD to focus their PDAs within the teaching improvement domain as a priority. Nonetheless, systematic implementation of PDAs will be required in diagnostic radiology PGME to ensure that all stakeholders possess the requisite skills associated with not only the teaching improvement domain but also all domains. 30 Preferred formats for PDAs pertaining to CBD in diagnostic radiology PGME have only been limitedly assessed. A recent needs assessment identified half-day academic retreats and educational lecture sessions or workshops as preferred formats for development of knowledge of CBD among diagnostic radiology residents. 31 Preference for such immersive formats has been suggested to be attributed to their interactive nature and the opportunity to foster discussion regarding the transition to CBD. 31  There are 2 main limitations to our study. First, our study is limited by its cross-sectional, survey-based design. Only faculty involved in the teaching of diagnostic radiology residents and diagnostic radiology residents were eligible to participate in our study and information was not collected from other stakeholders. Our study is also limited by inherent sampling bias as those with a predetermined opinion of CBD, whether positive or negative, may have increasingly chosen to participate in this study. Second, our study is limited by a low faculty response rate. This low response rate may demonstrate disinterest in CBD and is worrisome as it may portend poor "buy in" from faculty, which may be a challenge to the successful implementation of CBD in diagnostic radiology PGME. This limitation highlights the importance of evidencebased PDAs. Given this result, it may be wise to emphasize faculty PDAs in the implementation of CBD in diagnostic radiology PGME.

Conclusion
We evaluated faculty and residents' knowledge and perceptions of an established CBME model, CBD, in diagnostic radiology PGME and identified evidence-informed priorities for associated PDAs. We identified gaps in diagnostic radiology faculty and residents' knowledge and perceptions of CBD. Assessment of diagnostic radiology faculty and residents' knowledge and perceptions of CBD may help to inform the provision of associated PDAs. Despite the merits of CBME, CBME models may only be successfully implemented in diagnostic radiology PGME programs if stakeholders understand the principles and educational advantages of CBME over the current time-based PGME training paradigm. Our results and proposed CBD PDAs framework may help to guide diagnostic radiology PGME programs in designing evidence-informed PDAs to strengthen stakeholders' understanding of the principles and educational advantages of CBD, which may meaningfully contribute to the successful implementation of CBD or other CBME models in diagnostic radiology PGME.

Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.

Supplemental Material
Supplemental material for this article is available online.