Assessing research competency development in Canadian psychiatry residency programs: A systematic review and future directions

Objective: We aimed to conduct a systematic review to identify curricular and educational interventions to build research competency among Canadian psychiatry residents and fellows transitioning to the competency-by-design framework. Methods: The PRISMA guidelines were followed, searching five databases from their inception to February 2023 for relevant evaluation-type studies exploring research competency among psychiatry residents and fellows. We appraised thestudy’s quality using the Joanna Briggs Institute’s risk of bias tool for observational designs. Results: Overall, 36 original articles met our inclusion criteria. Surveys (n = 10) showed that participation in scholarly research, quality improvement, or educational projects relevant to psychiatry is needed in most residency programs. However, these vary significantly across programs; few need direct research experience for residency completion. The interventions spanned four categories: externally funded comprehensive research training programs (n = 5); resident research tracks (n = 11); workshops and seminars (n = 7); and specific modules (n = 3). Reported outcomes included overall program ratings, research output, and career trajectory. The quality of most studies was low because of the lack of controls or validated metrics for evaluating outcomes. Conclusions: While many studies have explored best practices in research curricula, the current literature does not inform competency-based models for Canadian psychiatry residency programs incorporating research training requirements. Further description is needed from Canadian psychiatric training bodies regarding appropriate curricula, milestones, and metrics for evaluating research competencies.


Introduction
5][6] Consequently, research training has become an increasingly mandated component of the psychiatry residency experience in the United States to address the critical shortage of research psychiatrists.
In recent years, the landscape of psychiatric residency training in Canada has undergone a significant transformation with the adoption of competency by design (CBD).The Royal College of Physicians and Surgeons of Canada (RCPSC) has emphasized the importance of research, dedicating a CanMEDS role to scholarly development.It has specifically outlined requirements during psychiatric residency training (Appendix 1).Within the CanMEDS Scholar role, psychiatry residents must understand the scientific principles of research and scholarly inquiry; contribute to the creation and dissemination of knowledge and procedures applicable to health; describe the principles of research, scholarly inquiry, and research ethics; pose a scholarly question; conduct a systematic search for evidence; select and apply appropriate methods to address the question; and disseminate the findings of a study.
As of 1 July 2020, Canadian psychiatry residency programs shifted to CBD, 7 which aims to prepare psychiatrists for practice by orienting training to competencies derived from the needs of society and patients. 8To operationalize CBD, the RCPSC has adopted "entrustable professional activities" (EPAs), 9 which are units of professional practice entrusted to trainees demonstrating the necessary competence to execute the activity independently. 10Working groups in psychiatry have defined 20 EPAs across four stages of residency training, with some for specific psychiatric subspecialties.  Althgh the RCPSC has described the research requirements and emphasized research as an essential component of psychiatric training, 35,36 the only EPA related to research in the current CBD framework is early in training within the Foundations of Discipline (FoD) stage (EPA #5), which specifies that two direct observations from at least two different observers are required of the resident to "perform a critical appraisal and presenting psychiatric literature."Key features of this EPA include: (1) "A focus on critical appraisal of literature to make appropriate clinical decisions and to encourage lifelong learning and acquisition of new knowledge and skills in the specialty"; (2) "Includes posing a clinically relevant question, performing a literature search, critically appraising the literature, and presenting in a group setting"; and (3) "Includes presentations such as grand rounds, journal club, case conference, morbidity and mortality rounds or quality improvement rounds." As CBD has progressed in Canadian psychiatry residency training, different program directors and competency chairs have noted issues with some of the EPAs and have been providing feedback to the Royal College.As the original cohort moves to different stages of their training, there is mounting pressure on programs to ensure that their residents achieve the current EPAs.However, the curriculum has become even more packed with the Royal College exam moving to the end of PGY4.Consequently, it may be even more difficult to address research competencies adequately.As EPAs tend to focus more on clinical skills that all generalist psychiatrists need, it is unlikely that there will be an expansion to have more research EPAs.However, other EPAs in the current CBD curriculum for psychiatry residents inherently touch on research competencies.For example, in the Core of Discipline stage, another EPA focuses on "teaching for students, residents, the public and other health care professionals."A key feature of this EPA includes "critical appraisal of relevant literature," but it is more applicable to presenting, where they also talk about adapting language and material to the audience's needs and effective presentation skills.
Regrettably, the research-related EPAs provide little description or guidance on what meets the standard for individual evaluations on critical appraisal and presenting findings.Furthermore, there are no standards set for how preceptors who will evaluate this EPA must themselves have recognized skills in critical appraisal and conducting research, which could lead to unreliable assessments across sites and preceptors.There is also no description of what formal critical appraisal skills need to be demonstrated and what level of sophistication is required to meet the standard.Thus, the requirements of the EPA appear to fall short of what the RCPSC requires. 37,38espite the RCPSC's recognition of the significance of research within CBD, the implementation and evaluation of research competencies in psychiatry residency programs remain complex and multifaceted.To address this, we conducted a systematic review of educational interventions related to research for psychiatry residents and fellows.The primary goal of this review is to clarify the requirements and standards for critical appraisal and presentation skills within the CBD framework.By evaluating and synthesizing existing educational interventions, we aim to provide guidance and support to Canadian psychiatric residency training programs, helping them develop a more consistent and effective approach to meeting the CanMEDS Scholar role's researchrelated expectations.This systematic review, therefore, plays a crucial role in enhancing the competency of psychiatry residents in research skills within the evolving CBD framework.

Protocol and registration
To provide a comprehensive synthesis of educational interventions related to research for psychiatry residents and fellows, we conducted a systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines. 39Our systematic review protocol was registered with PROSPERO (CRD42021244124).We have included a checklist with the relevant PRISMA items in Appendix 2.

Eligibility criteria
Eligibility criteria for the included studies were established based on the population-intervention-comparison-outcomestudy design (PICOS) framework, which guided our selection process 40

Selection of studies
Following the removal of duplicates, one coauthor (AB) completed two independent rounds of screening (first by title and abstract and then by full-text review).

Data extraction and management
We

Assessment of risk of bias in the included studies
2][43][44] Therefore, we assessed the study's quality using the Joanna Briggs Institute's (JBI) risk of bias tool for observational studies. 45,46In brief, the JBI checklist appraised eight domains of study quality in observational designs, including the description of study participants, study eligibility criteria, methods for measuring and defining exposure and outcome variables, accounting for confounding factors, and choice of analytic techniques.Each of these eight domains receives a 0 or 1, with higher scores indicating better quality.Studies receiving 6 or more points were designated "High quality"; otherwise, they were "Low quality."

Synthesis of the findings
We undertook a descriptive synthesis as a meta-analysis was not feasible due to excessive heterogeneity in the study design and outcome reporting, including overall program ratings (e.g., participant satisfaction, confidence with a skill), research output (e.g., publications, awards, grant funding, citations), and research career trajectory (e.g., transition or matriculation to further research training, academic positions, affiliation with professional bodies).However, we identified themes from surveys on psychiatry residents that provided a context for attitudes and perceived barriers to research.In addition, we identified potential strategies that residency programs could employ to enhance research competency with their residents from intervention-type studies.

Study quality
Using the JBI instrument, 34 of the 36 studies received lowquality ratings (Table 3).Most intervention-type studies reported numerous positive outcomes (e.g., grants, publications, matriculation to research roles) based on retrospective analyses but without any formal comparison to baseline productivity before implementing the program or to an external comparison group; only two of the studies 64,82 included a control group.

Synthesis of findings from the survey studies
While participation in scholarly research, quality improvement, or educational projects relevant to psychiatry is required in most residency programs, requirements vary significantly.Few require any direct research experience for residency completion.For example, a 2014 survey of representatives of all 17 accredited Canadian residency programs found that among the 11 programs mandating research requirements, qualifying activities ranged from fully independent resident-led research projects to quality improvement initiatives, assisting with faculty research, or simply presenting research. 54However, none required publication of the resident's final work. 54though most programs offer some form of research activity (e.g., at least presenting an article at a journal club), earlier survey studies demonstrated minimal research resources were available to residents. 47,48,50,515][56] Epidemiological methods, critical appraisal skills, ethical issues, and research design were the most commonly taught topics in didactic resident seminars. 495][56] Agreeing on a research training plan with a mentor and regularly reviewing support and supervision  (n = 54) The authors developed, implemented, and evaluated an interactive, webenhanced course to provide tools for conducting and reporting systematic reviews and meta-analyses There was overall participant satisfaction and improved confidence with reviews.Academic productivity included 11 presentations, four publications, and two awards through a newly developed mentoring process was associated with high trainee satisfaction levels. 52While early exposure to research was initially thought to promote future research interest, 55 two national surveys of Canadian psychiatry residents have challenged this assumption, suggesting that most residents lose interest in research by the time they graduate and that participation in a research track does not appear to enhance research interest. 55,56n summary, survey studies indicated that while most programs offered some research activity, the availability of resources (e.g., protected time), the content of didactic seminars, and the extent of research requirements varied substantially across programs.There were also challenges with enhancing research interest among residents, particularly those who lack pre-residency research training experience.

Synthesis of findings from the intervention studies
Across studies, several identified various mechanisms by which psychiatry residents could receive additional research training.At the highest end, we identified several externally  83 The CIP has been very successful, and most graduates who have gone through the program have become quite involved in research-although not all. 84,8573,79 Similarly, 3-day NIMH-funded psychiatry research conferences, which included morning plenary sessions and afternoon small-group teaching sessions, appeared to consolidate knowledge and provide practical research skills for resident participants. 86For child and adolescent psychiatry trainees, designated research workshops encouraged child psychiatrists who wished to pursue a research career and effectively encouraged trainee research. 87here were also three miscellaneous research training interventions.The first was a novel online research ethics training module for psychiatry residents, which incorporated five modules for teaching psychiatry residents about the basics of ethical concepts, including autonomy, benefits, and justice. 81The second was a systematic review and metaanalysis course, developed using an interactive, webenhanced module that gave trainees an overview of the essential tools for conducting and reporting systematic reviews and meta-analyses, overall participant satisfaction, and improved confidence with reviews. 77Finally, a residentled research newsletter showcasing resident research activity was associated with a nearly 50% improvement in research interest and participation in research projects among resident participants. 78iven the lack of evidence in the literature, we felt that including some expert opinion could be worthwhile, and consequently, offering some practical advice could be helpful.Based on these findings and our own experience, we compiled a list of resources, strategies, and "tips" for psychiatry residents interested in research (Appendix 4).

Summary of findings
The present study is the first comprehensive review of research training for psychiatry residents.Our review found 36 studies describing the state of psychiatry resident research opportunities and evaluating diverse research training methods.Formal research training programs enhanced the scientific workforce consistently, promoted future research involvement, and generated awards, publications, and grants.However, as these programs might simply sustain the scholarship of residents with previous research experience and interest, they may not appeal to trainees with less research interest or those who foresee non-research-based careers.
While there is insufficient evidence to inform guidelines for organizing research training programs concerning national requirements, we offer suggestions based on the available information. 74Different intervention approaches studied may help inform how Canadian residency training programs can develop a more uniform approach to meeting the CanMEDS Scholar role for research.To help set up a reliable standard, a pragmatic recommendation would be for all training programs to teach critical appraisal and presentation skills.As this EPA occurs early in training within the FoD stage, relevant teaching to achieve this EPA should cooccur during the corresponding phase of training (i.e., PGY1&2).For example, in-person research training workshops or journal clubs 88 or online training (i.e., the United Kingdom's Critical Appraisal Skills Program) could address critical appraisal content using checklists to help evaluate critical appraisal skill competence when residents present. 89n parallel, all Canadian psychiatry residency training programs should have a research requirement, yet only 11 of the 17 accredited programs reported including research requirements. 54At a minimum, Canadian psychiatry residents should complete a review of a major psychiatric topic applicable to their future practice and present it formally at grand rounds.The proposed requirement would demonstrate expertise in an area residents plan to practice and distinguish them from their peers for practice placement.
In a parallel vein, it is interesting to contrast the training of psychiatry residents with related mental health professionals' training.For example, a previous study compared the training of psychologists and psychiatrists, concluding that the latter have little formal training in studying human behavior and little contact with problems and methods in science.In contrast, the former has a more extensive background in studying human behavior and formal training in science. 90To that end, another study suggested that there may be mutual benefit from academic collaboration across institutions to achieve their respective educational and training missions. 91rengths and limitations.While augmenting the current research-oriented EPAs may be beyond the scope of this article, we believe this article has several strengths.First, based on existing literature, it describes the components and sophistication of critical appraisal and presentation skills that psychiatry residents need to meet for entrusting the related EPA.These descriptions provide Canadian programs with tangible suggestions for base program research requirements.Second, our review's findings identify the importance of distinguishing research capability and development levels for different residents.Different tracks could be available (i.e., basic for the general clinician, enhanced for the academic clinician, and advanced for the clinician-scientist/ research track academic).
However, a few limitations call for further discussion.First, as our search strategy focused on publications that focus on psychiatry, we excluded reviews on more general approaches.To address this, we included additional data on the RCPSC's CIP from a 2011 review published by Hayward et al. (performed by the RCPSC 13 years after the institution of the CIP in 1995) and a 2010/2011 survey on Canadian CIP programs by Hayward et al., 85 as these indirectly provided some coverage of the psychiatry programs that were involved.Finally, as the CIP has been a considerable success and some coauthors have had personal experience with this excellent RCPSC program, we discussed it in our review.
Second, despite the breadth of findings, the included studies' quality was poor due to few controls and standardized measurements.Given the uncontrolled designs of most studies, the presented findings are highly susceptible to selection bias.Most research tracks and formal training programs cater to motivated trainees. 53Even the two high-quality studies 64,82 had severe methodological limitations in selecting controls (i.e., these groups were not randomly allocated).
Third, no intervention studies or surveys addressed critical appraisal and presentation skills to inform Canadian psychiatry residency training programs about a standard to meet this EPA in the current CBD framework.There were few Canadian-based intervention studies, and most recommendations were extrapolated to Canadians.
A fourth limitation was the high heterogeneity in reported outcomes (precluding quantitative synthesis) and methods across studies-some studies not fully delineating the feasibility or effectiveness of their interventions and few measuring sustained competence over a long-term follow-up period were another limitation of our review.

Conclusions
In conclusion, our systematic review has shed light on the challenges of integrating research training within the CBD framework in psychiatry.The interventions described in the included studies primarily focused on cultivating productive, independent researchers, while CBD emphasizes the development of research basics, such as critical appraisal and research fundamentals, alongside clinical excellence.
Drawing insights from other successful CBD models, we propose that psychiatry research training could incorporate one or two general EPAs applicable to all residents, emphasizing foundational research competencies.Additionally, specialized or enhanced EPAs could be designed for residents pursuing research tracks or fellowships, allowing them to deepen their research expertise.
Establishing core metrics for evaluating research competency across Canadian psychiatry residency training programs is crucial.By doing so, the RCPSC and the Canadian Psychiatric Association can promote a standardized and comprehensive approach to research training, aligned with the principles of CBD.
In light of the limited evidence available in the literature, we hope that our compilation of resources, strategies, and tips for psychiatry residents interested in research will serve as a valuable, practical element of this article.These resources aim to empower aspiring researchers and contribute to the overall enhancement of research training within psychiatry residency programs.
While integrating research training within CBD presents challenges, we believe that our recommendations and the proposed framework will foster a culture of research excellence among psychiatry residents and contribute to advancing the field of psychiatric research in Canada.
on other boards.Marlon Danilewitz reports personal fees from Eisai Ltd and Winterlight Labs outside the submitted work.The remaining coauthors have no relevant disclosures.

Table 1 .
Characteristics of the survey studies (n = 10).Residents identified research as an essential part of psychiatry, and most welcomed the idea of formal training.However, two-thirds felt that their departments were underproductive in terms of research U.S.: United States.

Table 3 .
Joanna Briggs Institute's risk of bias assessments.