Medical Student Portfolios: A Systematic Scoping Review

Phenomenon Medical Student Portfolios (MSP)s allow medical students to reflect and better appreciate their clinical, research and academic experiences which promotes their individual personal and professional development. However, differences in adoption rate, content design and practice setting create significant variability in their employ. With MSPs increasingly used to evaluate professional competencies and the student's professional identity formation (PIF), this has become an area of concern. Approach We adopt Krishna’s Systematic Evidence-Based Approach to carry out a Systematic Scoping Review (SSR in SEBA) on MSPs. The structured search process of six databases, concurrent use of thematic and content analysis in the Split Approach and comparisons of the themes and categories with the tabulated summaries of included articles in the Jigsaw Perspective and Funnelling Process offers enhanced transparency and reproducibility to this review. Findings The research team retrieved 14501 abstracts, reviewed 779 full-text articles and included 96 articles. Similarities between the themes, categories and tabulated summaries allowed the identification of the following funnelled domains: Purpose of MSPs, Content and structure of MSPs, Strengths and limitations of MSPs, Methods to improve MSPs, and Use of E-portfolios. Insights Variability in the employ of MSPs arise as a result of a failure to recognise its different roles and uses. Here we propose additional roles of MSPs, in particular, building on a consistent set of content materials and assessments of milestones called micro-competencies. Whislt generalised micro-competencies assess achievement of general milestones expected of all medical students, personalised micro-competencies record attainment of particular skills, knowledge and attitudes balanced against the medical student’s abilities, context and needs. This combination of micro-competencies in a consistent framework promises a holistic, authentic and longitudinal perspective of the medical student’s development and maturing PIF.


Introduction
At a time when medical education is embracing a more personalised approach to knowledge attainment, skills training and development of professional behaviours, portfolios promise a means for medical students to better understand, reflect upon and actively shape their learning and development 1 . Complementing traditional assessment methods with wider longitudinal appraisals of an individual's growth, portfolios add a personalised dimension to logbooks 4,5 , by serving as a repository for written examinations, tutor-rating reports and bedside assessments 6 as well as individual reflections and analyses.
Indeed, portfolios offer medical students "a self-regulated, cyclical process in which [they may] mentally revisit their actions, analyse them, cogitate alternatives, [and] try out alternatives in practice" 7 . It is this platform to showcase individual educational, research, ethical, personal and professional development 1,8 , and guide specific, holistic and timely feedback and remediation throughout the individual's medical education that underscores growing interest in portfolio use among medical students (henceforth medical student portfolios or MSPs) 4,12 . However, despite their growing traction 13 , MSPs show significant variability in their structure and content. With local, practical, sociocultural, educational and healthcare considerations prioritising different types of data, the role of MSPs remains limited.

Need for the Review
With MSPs representing a sustainable and effective educational undertaking that provides insight into the medical student's development, needs, values and beliefs that may guide their professional identity formation (PIF), better understanding of the principles behind their use, the key elements within them and a framework for consistent utilisation is required.

Methods
To determine what is known about MSPs, a systematic scoping review (SSR) is proposed to study current literature to enhance understanding of their roles and structure. These insights will also help guide the design of a consistent framework for MSPs to be used across different settings, purposes and specialities given their ability to evaluate data 14 from "various methodological and epistemological traditions" 19 .
To overcome SSR's variable methodological steps, guidance and standards, this review adopts the Systematic Evidence Based Approach (SEBA) 20 . A SEBA guided SSR (henceforth SSR in SEBA) facilitates the synthesis of an evidence-based, accountable, transparent, and reproducible analysis and discussion.
Steering this process and boosting accountability, oversight, and transparency, this SSR in SEBA sees an expert team involved in all stages of this review. The expert team comprised of medical librarians, local educational experts, and clinicians.
SSRs in SEBA are built on a constructivist perspective acknowledging the personalised, reflective, and experiential aspect of medical education and recognising the influence of particular clinical, academic, personal, research, professional, ethical, psychosocial, emotional, legal and educational factors upon the medical student's learning journey, professional development and personal growth 27 .
To operationalise the SSR in SEBA, the research team adopted the principles of interpretivist analysis to enhance reflexivity and discussions 18,32 in the six stages outlined in Figure 1.
(Insert Figure 1. The SEBA Process) Stage 1 of SEBA: Systematic Approach 1. Determining the title and background of the review The expert and research teams determined the overall goals of the SSR and the population, context and concept to be evaluated.

Identifying the research question
Guided by the PCC (population, concept and context), the expert and research teams agreed upon the research questions. The primary research question was "what is known about medical student portfolios?". The secondary questions were "what are the components of MSPs?", "how are MSPs implemented?" and "what are the strengths and weaknesses of MSPs?".

Inclusion criteria
All peer reviewed articles, reviews and grey literature published from first January 2000 to 31 st June 2021 were included in the PCC and a PICOS format was adopted to guide the research processes 35,36 . The PICOS format is found in Table 1.

Searching
A search on six bibliographic databases (PubMed, Embase, PsycINFO, ERIC, Google Scholar and Scopus) was carried out between first to 10 th September 2021. Limiting the inclusion criteria was in keeping with Pham et al's (2014) approach to ensuring a sustainable research process 37 . The search process adopted was structured along the processes set out by systematic reviews.

Extracting and charting
Using an abstract screening tool, members of the research team independently reviewed the titles and abstracts identified by each database to identify the final list of articles to be reviewed. Sambunjak et al's (2010) approach to 'negotiated consensual validation' was used to achieve consensus on the

Stage 2 of SEBA: Split Approach
Three teams of researchers simultaneously and independently reviewed the included full-text articles. Here, the combination of independent reviews by the various members of the research teams using two different methods of analysis provided triangulation 41 , while detailing the analytical process improved audits and enhanced the authenticity of the research 42 .
The first team summarised and tabulated the included fulltext articles in keeping with recommendations drawn from Wong et al's (2013) "RAMESES publication standards: metanarrative reviews" 43 and Popay et al's (2006) "Guidance on the conduct of narrative synthesis in systematic reviews" 44 . The tabulated summaries served to ensure that key aspects of the included articles were not lost (Supplementary File 1).
Concurrently, the second team of three trained reviewers analysed the included articles using Braun & Clarke's (2006) approach to thematic analysis 45 . In phase one, the research team carried out independent reviews, actively reading the included articles to find meaning and patterns in the data. In phase two, 'codes' were constructed from the 'surface' meaning and collated into a code book to code and analyse the rest of the articles using an iterative step-by-step process. As new codes emerged, these were associated with previous codes and concepts. In phase three, the categories were organised into themes that best depict the data. An inductive approach allowed themes to be "defined from the raw data without any predetermined classification". In phase four, the themes were refined to best represent the whole data set. In phase five, the research team discussed the results of their independent analysis online and at reviewer meetings. 'Negotiated consensual validation' was used to determine a final list of themes.
A third team of three trained researchers employed Hsieh & Shannon's approach to directed content analysis and independently analysed the included articles 46 . This analysis using involved "identifying and operationalising a priori coding categories". The first stage saw the research team draw categories from Davis et al.'s (2001) "AMEE Medical Education Guide No. 24: Portfolios as a method of student assessment" 47 to guide the coding of the articles. Data not captured by these codes were assigned a new code in keeping with deductive category application. Categories were reviewed and revised as required. In the third stage, they discussed their findings online to achieve consensus on the final codes.

Stage 3 of SEBA: Jigsaw Perspective
As part of the reiterative process, the themes and categories identified were discussed with the expert team. Here, the themes and categories were viewed as pieces of a jigsaw puzzle and areas of overlap allowed these pieces to be combined to create a wider/holistic view of the overlying data. The combined themes and categories are referred to as themes/categories. Creating themes/categories relied on use of Phases 4 to 6 of France et al.'s (2016) adaptation 48 of Noblit and Hare's (1998) seven phases of meta-ethnography 52 . To begin, the themes and categories were contextualised by reviewing them against the primary codes and subcategories and/or subthemes they were drawn from. Reciprocal translation was used to determine if the themes and categories could be used interchangeably.

Stage 4 of SEBA: Funnelling Process
To provide structure to the Funnelling Process, we employed Phases 3 to 5 of the adaptation. We described the nature, main findings, and conclusions of the articles. These descriptions were compared with the tabulated summaries. Adapting Phase 5, reciprocal translation was used to juxtapose the themes/categories identified in the Jigsaw Perspective with the key messages identified in the summaries. These verified themes/categories then form the line of argument in the discussion synthesis.  Table 2 for ease of review.
Funnelled Domain 2: Content and structure of MSPs

Content in MSPs
Similarly, discussions on the contents of MSPs are limited and have been summarised in Table 3. The content can be broadly categorised into content provided by the institution, medical students, and feedback/assessments by other stakeholders.

Structure of MSPs
Standardisation within and across portfolios may be achieved through the use of a clear template 4 or set of guidelines 53 . MSPs with clear delineation of contents required 54 were found to boost student receptivity 55,56 and enhanced reliability and validity during portfolio assessment 47,55,57 .
However, a flexible approach allowing medical students to personalise their MSPs 58 and express themselves more freely 59 facilitates portfolio student-centricity 60,61 and ownership 53 . By encouraging students to incorporate their own content, such as reflective diary entries 55 , reflective essays 57 , video recordings 58 , audio recordings 59 , poetry or art 62 , improvements may be seen in the quantity and quality of their reflections 56 .

Funnelled Domain 3: Strengths and Limitations of MSPs
Given the lack of elaboration, much of the data for this domain is summarised in tables to aid easy review.

Strengths
Strengths of MSPs are highlighted in Table 4.

Limitations
The limitations of MSPs are highlighted in Table 5.

Funnelled Domain 4: Methods to Improve MSPs
The potential methods to improve MSPs are highlighted in Table 6.

Stage 5 of SEBA: Analysis of Evidence-Based and Non-Data Driven Literature
Evidence-based data from bibliographic databases were separated from grey literature such as opinion pieces, perspectives, editorial, letters and non-data based articles drawn from bibliographic databases and both groups were thematically analysed separately. The themes from both groups were compared to determine if there were additional themes in the non-data driven sources that could influence the narrative. In this review, the themes from the two data sources overlap, suggesting no undue influence upon the findings of this review.

Stage 6 of SEBA: Synthesis of SSR in SEBA
The narrative produced from consolidation of the funnelled domains was guided by the Best Evidence Medical Education (BEME) Collaboration guide 89

Discussion
In answering its primary and secondary research questions, this SSR in SEBA reveals that MSPs have expanded beyond merely repositories of assessments and are now seen as a means of triangulating and contextualising assessments and their impact upon individual medical students. MSPs also allow students, faculty, and institutions to better understand the medical student's needs, abilities, expectations, and aspirations, aiding the provision of personalised mentoring and remediation. However, to meet these wider roles, manageable 87 and "authentic" portfolios that improve levels of engagement 91 are key. Here, authenticity refers to the "extent to which the outcomes measured represent appropriate, meaningful, significant and worthwhile forms of human accomplishments" 47 and serves to enhance the trustworthiness of what is largely qualitative data, and the validity of longitudinal assessments that help to map the development of their clinical competency 4 and professional identity formation 4,12,92 .
However, current MSPs lack a consistent structure. While broad commonalities including learning objectives and professional expectations and roles to be met, and reflections, learning activities, self-assessments, achievements, and other evidence of competencies, MSPs vary significantly in their focus and content. Yet, these variations and particularities are unsurprising given the different practice settings, structure and program goals established by the host institution. These differences underpin the presence of different types, "depth" and nature of content prioritised. Inherent variability brought about by personalisation of longitudinal data, "choice of materials by the student" 54 and "individualised selection of evidence" 47 , ultimately limits the use of portfolios beyond the confines of a specific institution. This lack of consistency raises concerns about the efficacy of MSPs in providing a holistic perspective of the medical student's personal, academic, clinical, and professional development.
We believe that these concerns may be bridged in part by harnessing the ability of current MSPs to capture education and assessment in specific areas of practice. Our findings suggest that current MSPs encapsulate several entrustable professional activities (EPA)s 94 . Each EPA however shares common aspects of other EPAs that may not be directly contained within a particular MSP. We believe that it is possible to harness these overlapping aspects to make MSPs more widely applicable. Here, we build upon the notion that micro-credentialling that incorporates "circumscribed assessments" of a specific EPA, such as "interpreting and communicating results of common diagnostic and screening tests", may be extrapolated to other EPAs such as "[communicating] in difficult situations" in a different practice setting 97   Guides remediation plans for underperforming students 1,62,91,105,111,116,135,140,142 • Specific to summative portfolio assessment: ○ Ensures that students take the portfolio exercise seriously 57,114 ○ Students will be spurred on to improve themselves should they receive negative feedback 75 ○ Better demonstrates achievement in competencies such as professionalism, teamwork, and communication skills 111 • Specific to formative portfolio assessment: ○ Enables constant improvement through feedback and reflection 6,7,60,71,75,105,116,127,133,140 ○ Fosters self-motivation 5,69 and intrinsic motivation to reflect 91,115 .

Others
• Encourages students to discuss their private thoughts 103 • Prepares students for postgraduate work ○ Easily transferable when needed in the future 80 to facilitate job applications 103,104 or acquisition of letters of recommendation for future training 80 ○ Helps to ease transition to postgraduate educational practice 74 as portfolios and portfolio assessment are often utilised at postgraduate level 55 • Improves teaching within undergraduate programs ○ Improves faculty's understanding of students Better understand students' thinking and attitudes 65 Directs discussion during meetings with advisees 65,74 ○ Identifies gaps in the curriculum 56,101 such as through providing opportunities for students to evaluate teaching activities 56 • Helping students to develop better rapport with others including patients 62,118,122 , clinical teams 62 and other students 132 8 Journal of Medical Education and Curricular Development small, professional learning milestones that all students need to attain before proceeding to the next competency-based stage. These are requisite knowledge, skills and attitudes all soon-to-be clinicians must have. Personalised micro-competencies, in turn, are determined by the individual's particular goals, training, abilities, skills and experiences. They are determined by the medical student and tutors and must be consistent with institutional codes of conduct and expectations.

Inauthentic
Provide only vignettes of a student's journey 59 , and students may hide evidence of their weaknesses 54,59,63,70,104,126 , fail to express their authentic views 63 or even fabricate reflections 78 They may also perform poorly under stress during assessments included in their portfolios such as directly observed work-based assessments 59,137 Students tend to have a poor self-assessment capacity 72,111,151 Perceived quality of portfolio relies heavily on the individual's reflective ability 55,105,121 which is unfavourable for students with poor reflective skills ○ Subjective Students may create their portfolios differently based on their own interpretation of the purpose of the portfolio 59 Student's portfolios may unknowingly be judged on irrelevant aspects such as layout and format 4 This may be amplified if student identity is not anonymised to examiners evaluating the portfolios 119 ○ Overly structured 47,53,57,59,62,64,119 Highly structured portfolios with a rigid format can lead to students including less of their personal observations and reflections, which diminishes the portfolio's capacity for authentic assessment of the student and their development • Problematic assessment process ○ Poor student understanding 11,53,62,63,73,104,116 ○ Time consuming There may be insufficient time for comprehensive assessments in the clinical setting as taking time to assess students must be balanced with providing quality patient care 59 Time consuming for assessors 1,5,11,13,53,55,60,63,65,68,74,104,112,116,140 Human resource intensive 6,112,137,140 Excessive paperwork 1,55,74,106 ○ Lack of standardisation among examiners Poorly standardised assessment procedure leads to poor consensus among assessors 117 • Lack of training for assessors limits the use of work-based assessments within portfolios for assessing student competence 137 Portfolio Implementation  64 Failure to understand role as portfolio mentors 64,110 Did not engage in reflection personally 64 Difficulty finding methods to help students 78 Poor impression of portfolios and their role in education 66,78 Poor relationship with student 103 Table 6. Methods to improve MSPs.

Increase Mentorship
Mentorship refers to a system where students are assigned to faculty throughout their training and portfolio creation to coach them 54,57,101 , engage them in supportive dialogue 63,64,108,118,148 , provide feedback 1,61,63,64,133 and encourage them to fully engage with their portfolios 74,78,103,131,146 .
Benefits of Mentorship • Crucial to portfolio success 4,7,63,64,78,79,87,104,131 because it helps guide the students' reflective process 57,65,131,146 , enhances learning 1,57,74,135 and increases student receptivity towards their use 7,64,103 Improving quality of mentorship • Train mentors 66,78,87,123 and utilise verified teaching methods that foster reflection 152 and ensure mentors are able to stretch their students in their reflective practice 78 • Recruit good mentors ○ Willing to engage students 108 ○ Understands reflection 129 and their responsibility to teach students how to utilise reflections purposefully 79 ○ Able to build trust and rapport with students 64 Having a structured mentoring programme to guide portfolio use • Some institutions encourage frequent weekly meetings with mentees 108 , while others believe that mentorship can occur as infrequently as two to three times a year 4,57,64 • Keep the student to mentor ratio small such as having one-to-one interactions 6,70,79 Encourage portfolio uptake  4,7,53,54,56,57,60,64,65,70,91,102,114,121,123 Increase Exposure • Students who had been exposed to them for some time 6,91 had more positive attitudes towards portfolios.
○ Embed portfolio into the curriculum 54,64,72,104 and encourage faculty and department staff to reference it in daily practice 77 ○ Early portfolio introduction 54,129 Structure portfolio appropriately Organise portfolio based on its purpose • Organise the portfolio based on its purpose 125 .
○ For a portfolio focused on enhancing learning, the portfolio should include more self-reflection 54,56 and reasoned tasks that demonstrate student learning 56 . Improving portfolio assessment process Enhance learning through assessment process • Focus assessment on promoting student development 88 through providing useful feedback 121,124 • Enhance reflective learning • Ensure assessment does not compromise reflection 54 • Assess students based on the authenticity of their reflections 53 • Institute a central committee to review assessments and ensure ample learning experiences and assessment evidence exist to guide student learning 70

Journal of Medical Education and Curricular Development
They underscore the importance of assessing the student's individual needs and circumstances which influence which in turn shape the kind of training and support proffered. With expectations differing across practice settings and levels of training, both generalised andpersonalised micro-competencies must be clearly conveyed to the medical student and tutors in a timelyand structured manner. To encapture their learning and attainment, MSPs must forward clear learning plans to align expectations with evidence of diverse learning activities, reflective prompts and diaries, multisource formative and summative evaluations via standardised assessment tools and constructive feedback. These standardised baseline guidelines will lend clarity to portfolio developers and users. This may boost the latter's trust and receptivity towards regular portfolio use 55,56 . We believe that structured and consistent micro-certification of micro-competencies could be extrapolated beyond the initial goals of the MSPs and could provide a longitudinal perspective of the medical student's development. This is especially useful when considering competencies such as interpersonal, communication skills and systems-based practices. Perhaps here, too, the silver lining to changes in medical education practices due to the COVID-19 pandemic can be harnessed.
With many institutions incorporating online learning, e-portfolios should be institutionally sanctioned 85 with a dedicated team of portfolio developers and invested faculty members onboarding and overseeing their implementation. These considerations foreground the need for orientation sessions 10,62,64,67,104 to educate students and faculty on the identified EPAs as well as the use of generalised and personalised micro-competencies to ensure learning and assessment congruity and objectivity 91,105,106 . Embedding the portfolios into the formal curricula, assigning students mentors trained in reflective engagement, and establishing protected time for regular portfolio reviews would help to facilitate their consistent usage. Concurrently, portfolio use must be part of a continuous quality improvement process, building on feedback 107 and lessons learnt to promote further improvement to MSPs and portfolio assessment 10,11,47,62,78 . Indeed, both forms of micro-competencies underline the need for effective recording and oversight. This is especially important when micro-competencies provide a holistic appraisal of the medical student's progress and achievements, needs and abilities and provides insights into their professional identity formation. Capturing this data in a comprehensive, longitudinal manner replete with the medical student's reflections reveals a new dimension to portfolio use.

Limitations
Firstly, the review is limited by the omission of articles not published in English. This creates the risk of missing key papers. Furthermore, the focus on papers published in English led to focus on studies in North America and Europe. Secondly, while the articles comment on the sentiment of users including medical students on the effectiveness of portfolios for learning and assessment, there are a limited number of articles highlighting the perspectives of doctors who previously undertook the task of undergraduate portfolios. Hence, the review is limited by its inability to assess the long-term effectiveness and acceptability of portfolio usage after medical students enter the workforce as practicing medical professionals.  129 • Increase number of assessment points such as by adopting more work-based assessments within the portfolio 137 • Reduce subjectivity of assessment ○ Create and validate clear rubrics to assist assessors in their grading of students 121 ○ Increase number of assessors to achieve better inter-rater reliability 62,72,112,121 ○ Provide training to assessors 4,53,62,64,67,68,74,85,87,104,111,121,124,135 ○ Providing opportunities for discussion or feedback between assessors 4,8,63,72,105,111,116,117,124 • Introduce portfolio interviews where students can discuss and elaborate upon their portfolios personally 4,8,53,72,105,116,140 or even assess their own portfolios 5,55 Improve self-assessment process • Encourage students to include evidence to support their self-assessments to reduce inaccurate selfassessments 111

Conclusion
This SSR in SEBA reveals that if portfolios are to remain relevant and maintain their user-friendliness and accessibility, the future of MSPs must lie in improving assessments and in enhancing the manner in which they are designed.
While it is clear that assessments tools need to be enhanced to meet new perspectives of education and training, it is perhaps timely that this SSR in SEBA suggests key changes to portfolio use. In adopting e-portfolios for its accessible and expansive potential, it is clear that a robust and well-supported platform is critical. This platform ought to accommodate all manner of data and assessment results and remain a comprehensive repository of data. Categorised into different, sometimes overlapping, domains, data from this repository may be drawn to populate different designs of MSPs. Changing from one goal to another should therefore be simple. Such flexibility will still allow medical students to personalise their e-portfolios in a manner that they feel best represents their development without compromising faculty evaluation. A flexible yet robust e-portfolio such as this will also enable collaborations and facilitate input of corroborative data from third parties where required.
Moving forward, further research may be undertaken to identify the long-term effects of portfolio usage, the manner that portfolios are evaluated, and the impact it has on professional identity formation throughout and beyond medical school.

Glossary Terms
Professional Identity Formation An adaptive developmental process that involves the psychological development of an individual, and the socialisation of the individual into appropriate roles and participation at work.
Krishna's Systematic Evidence-Based Approach (SEBA) A structured and accountable approach used to guide analyses to ensure reproducible and robust data.

Split Approach
Combines content and thematic analysis of data to enhance the trustworthiness and depth of an analysis.

Jigsaw Perspective
Comparing overlaps between the themes and categories delineated by content and thematic analysis are considered in tandem, like complementary 'pieces of the jigsaw'. This allows for holistic perspective of data.