Studies within a trial priorities to improve the evidence to inform recruitment and retention practice in clinical trials

Background Trial execution commonly relies on experience and judgement because there is a lack of evidence to inform how best to design and deliver clinical trials. Recruitment and retention are critical determinants to trial success have been persistent challenges that impact various stakeholders including funders, researchers, and the public. Studies within a trial (SWATs) are a way to discover best practices for recruitment and retention strategies, however, the current SWAT landscape has not been formally explored to date. This study aimed to (i) identify where current activity is taking place (ii) understand if SWATs are addressing PRioRiTY questions (iii) highlight gaps in the literature for future research. Methods In November 2020, registered SWATs in the SWAT repository store were extracted and categorised into ‘recruitment’, ‘retention’ or ‘other’ based on the primary outcome. Recruitment and retention SWATs were subsequently mapped against PRioRiTY 1 and 2 questions and descriptive statistics were used to present the findings. Results 125 registered SWATs were extracted from the repository. 50 and 36 SWATs reported recruitment and retention as their primary outcome, respectively. A majority of recruitment SWATs investigated what and how information should be designed and delivered to potential trial participants (n = 23, 46%) and the advantages and disadvantages of using technology during the recruitment process (n = 9, 18%). Three of the Top 10 PRioRiTY 1 questions had no SWATs mapped against them. A majority of retention SWATs focused on the best ways to encourage participants to complete trial tasks (n = 24, 67%), how incentives should be implemented (n = 10, 28%) and strategies to make participants feel valued (n = 9, 25%). Five of the Top 10 PRioRiTY 2 questions had no SWATs mapped against them. Conclusions This study identified a mismatch between registered SWAT activity and the priority questions in recruitment and retention. Trial teams should consider the PRioRiTy 1 and 2 questions for recruitment and retention, respectively, when designing a SWAT. In addition, there is a great breadth of research taking place, but replication of existing research is needed to produce confident evidence-based guidance for trialists and researchers to implement into their work.


Background
Randomised trials are essential to provide evidence that advances clinical care and informs clinical practice guidelines. How the trials themselves are run, however, can be inefficient and sub-optimal. 'There is a peculiar paradox that exists in trial execution -we perform clinical trials to generate evidence to improve patient outcomes; however, we conduct clinical trials like anecdotal medicine: (1) we do what we think works; (2) we rely on experience and judgement and (3) limited data to support best practices.' Monica Shah, quoted in Gheorghiade et al. 2014 1 The conduct of clinical trials commonly relies on experience and judgement. It does so because there is no alternative: the evidence available to inform decisions is either limited or non-existent. This might not matter if we knew trials to already be highly efficient but as anyone who works in trials knows, this is not the case. 2 This disconnect between evidence (not much) and practice (decisions that need to be made every day) highlights the need to research how best to conduct clinical trials. This lack of evidence on how best to approach trials spans the entire clinical trial lifecycle from site selection to the dissemination of results.
Studies within a trial (SWAT) are used as a way to investigate and test alternative ways of delivering a trial process to generate evidence to support decisions about that process. A SWAT is defined as "a self-contained research study that has been embedded within a host trial with the aim of evaluating or exploring alternative ways of delivering or organising a particular trial process". 3 SWATs can be randomised evaluations, non-randomised evaluations, or more exploratory studies using qualitative methods to provide an answer to a given question. The process they test may be at any point of the trial life cycle and target anyone involved in the trial endeavour, such as participants, trial staff, clinicians, regulators, etc.
Recruitment and retention are two of the thorniest trial process challenges. [4][5][6][7] Participation in trials is dependent on the willingness of people, be it patients, the public or professionals, to volunteer their time and energy to the trial. It is not just the initial decision to participate; the commitment to the trial often needs to be sustained over long periods and may involve multiple questionnaires and/or follow up visits. As well as diminishing certainty, inadequate trial recruitment and retention can also raise ethical concerns for those already recruited into a trial in which the objectives cannot now be achieved. 8,9 Understandably, recruitment and retention have been identified as the top methodological priorities for trial methodology research within the UK. 10 Which aspects of trial recruitment and retention were most in need of research attention has been unclear. However, two recent prioritisation processes, Prioritising Recruitment in Randomised Trials Study (PRioRiTy I, 2018 11 ) and Prioritising Retention in Randomised Trials Study (PRioRiTy II, 2019 12 ), have changed this. The PRioRiTY 1 and 2 projects were completed as collaborative projects with the public, carers, and professionals to identify and prioritise unanswered questions for trial recruitment and retention respectively, which help direct research teams to create evidence that is important and can be applied in everyday trial conduct. The research agendas produced by the prioritisation exercises now require SWATs to help fill the evidence gaps.
To understand the current landscape of recruitment and retention SWATs registered on the SWAT repository (http:// www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrials MethodologyResearch/SWATSWARInformation/Repositories/ SWATStore/), we conducted a gap analysis to identify where efforts are currently focussed and, critically, where there are gaps. We had a particular focus on where effort is required to address the PRioRiTy projects Top 20 questions.

Methods
In November 2020 we extracted data on the title, lead contact, intervention and the status (i.e. active, inactive, completed) of the SWATs registered in the SWAT Repository Store. 13 The registered SWATs were organised into one of three groups, 'recruitment', 'retention' or 'other' based on the primary outcome stated in the SWAT protocol. Any studies listed on the SWAT repository that were not focused on recruitment, retention or not explicitly set within clinical trials were excluded.
The included SWATs were then mapped against the research questions that came from the recruitment and retention PRioRiTy projects (https://priorityresearch.ie). Mapping was completed by comparing the recruitment SWATs with PRioRiTy 1 questions, and retention SWATs with PRioRiTy 2 questions. Where a recruitment and retention SWAT could not be mapped onto a PRioRiTy question, the research team developed 'codes' for new categories. SWATs were mapped to all PRioRiTy questions considered relevant.
One researcher conducted the initial mapping independently (CB for recruitment and KG for retention) with subsequent verification of findings through discussion between three researchers (CB, KG, ST). The final complete mapping document is available in Supplementary File 1. Results are presented descriptively using frequencies.

Results
At the time of mapping (Novembber 2020), there were 125 SWATs listed on the database, with 86 of these reporting a primary outcome that focussed on recruitment or retention and was set within a trial. Of the 86, 50 SWATs considered recruitment and 36 SWATs retention, with 38 of the recruitment SWATs also proposing to evaluate retention as a secondary outcome. A total of 13 SWATs were inactive (i.e. had not started), 24 were ongoing, 41 had reached completion and the status of the remaining 8 SWATs was unclear. Reasons for inactive registrations is likely to be for one of two reasons. Firstly, the repository is a place where protocols can be posted for others to use, and it may not be the intention of the team to conduct the SWAT. Secondly, some SWATs were delayed because of delays faced by the host trial.
Of the 50 recruitment SWATs, 5 did not map onto a PRioRiTy I Question and 22 SWATs mapped onto more than one PRioRiTy I Question (e.g. both Question 4 and Question 6).The remaining 45 mapped SWATs aiming to improve recruitment primarily focused on two broad areas: 1. What and how information should be designed and delivered to potential trial participants (n = 23, 46%), which mapped to PRioRiTy I Question 4 (n = 14, 28%), and Question 2 (n = 9, 18%); 2. Advantages and disadvantages of using technology during the recruitment process (n = 9, 18%), which mapped to PRioRiTy I Question 10) ( Table 1).
Five SWATs with recruitment as a primary outcome could not be mapped to any of the PRioRiTy I questions and alternatively were categorised as 'site engagement' which involved interaction with a recruiting site (e.g. PI visiting site, use of site recruitment plan). Of the five unmapped SWATs, 1 was a protocol, 2 were inactive due to unopened host trials, 1 is unknown due to no response from the registered SWAT contact, and 1 is published.
Three of the Top 10 PRioRiTy I questions (Question 1, 4 and 7) had no recruitment SWATs mapped against them.
Of the 36 retention SWATs, all were mapped onto PRioRiTy II questions (Table 2), with 23 SWATs mapping onto more than one question (e.g. Both Question 4 and Question 19). A majority of SWATs mapped onto Question 4 (n = 24, 67%) which considered the best ways to encourage participants to complete trial tasks. The other top areas of retention activity cover PRIORITY II questions 18 and 19, on strategies to make participants feel valued and how incentives should be implemented.
Five of the retention PRioRiTY II Top 10 questions had no SWATs mapped against them and included areas such as what motivates a participants decision to participate, use of routine data, and patient and public involvement (See Table  2).

Discussion
Most SWAT activity in both recruitment and retention is focused on a small number of PRioRiTY 1 and 2 questions. This might not be a problem (some focus is a good thing) but there remains much variability within SWATs that appear to be answering the same PRioRiTy question. For example, there is considerable intervention heterogeneity in studies exploring monetary incentives in terms of value (e.g. £5-25) and mode (e.g. voucher, money). There is also a large focus on the way information is provided to participants for recruitment, and also the completion of postal questionnaires to encourage retention, but large variability exists within the repository meaning that conclusive guidance for research conduct cannot be drawn from the evidence.
This variability makes it hard to provide a definitive answer as to whether these interventions are effective at improving recruitment and/or retention, and efforts should be made to work collaboratively and create a strong evidence base by replicating what has been registered or coordinating efforts with trial conduct methodologists. Moreover, there is little evidence on specific populations such as younger or older adults and diverse ethnicities to help trialists select strategies most applicable to their own trial. A concerted effort of multiple evaluations of identical interventions at scale in a range of populations is needed.
The mapping exercise highlights gaps where further research is needed (Tables 1 and 2). Unexplored gaps include how to maximise the potential of routine clinical care for trial recruitment and retention (PRioRiTy I Question 1 and PRioRiTy Question 2). Additionally, there are several gaps in the Top 10 questions exclusive to recruitment or retention that highlight areas for innovation and may be best suited to an exploratory design using qualitative methods, for example, what motivates a participant's decision to complete a clinical trial and what strategies make participants feel valued? Qualitative enquiry is a means to explore the in-depth views and experiences of participants which may not otherwise be fully captured with quantitative methods alone.
Future consideration should be given to exploring how challenges or motivators to trial participation (both initial and continued) reported by participants can be used to drive the design of SWAT interventions for randomised evaluation. Approaching SWATs in this manner would strengthen existing evidence through the use of sequential research design methods where, for example, an initial qualitative study informs the need for further quantitative investigation and vice versa. An example of where this can be applied is conditional altruism, which describes the willingness to help others but with some perceived benefit to self, and has been identified as an influential reason for trial participation.14 Yet, there are no SWATs registered that target this influence to improve recruitment. Conversely, there are many studies on the use of monetary incentives, but no qualitative evidence to prove if, how and why it is a powerful motivator for the return of outcome data. Only one registered SWAT planned to use qualitative methods to investigate one of the PRioRiTy questions (PRioRiTy 1, Question 5). The absence of qualitative studies from the repository highlights that more discussion of SWATs with qualitative researchers would be helpful, as would the broader conceptualisation of SWATs as including qualitative work.
It should be recalled that the mapping exercise focused on SWATs registered on the SWAT Repository and did not encompass research publicised outside of this, meaning there may be existing research on the PRioRiTy gaps (e.g. Crocker et al., 2018 systematic review and meta-analysis of PPI involvement). This means that this SWAT mapping exercise is not an exhaustive list of all research on recruitment and retention and that some of our gaps may actually have some existing evidence to replicate/build upon. Those wishing to design and run SWATs should consult relevant, systematic reviews, or conduct their own, in addition to looking at the repository to ensure they are fully informed of the current evidence.
There any now many opportunities and resources available to support collaborative SWAT evaluations through the PROMETHEUS team (https://www.york.ac.uk/ healthsciences/research/trials/research/swats/prometheus/) and Trial Forge SWAT Network (https://www.trialforge.org/  9. What are the best approaches to optimise the informed consent process to 1improve recruitment of members of the public to randomised trials? 1 (2%) 10. What are the advantages and disadvantages to using technology during the recruitment process? 9 (18%) 11. How best can clinicians and other health professionals be educated to optimise recruitment to randomised trials?

(10%)
12. Does feasibility testing of recruitment procedure lead to improvements in recruitment? 0 (0%) 13. How can eligibility criteria for trial participants be optimised so that they ad recruitment to randomised trials? 0 (0%) 14. What is the value of making trials participants feel appreciated in being recruited to a trial and how can this be best achieved? 1 (2%) 15. What are the best approaches to ensuring manageable workloads for clinical and research staff responsible for recruiting members of the public to randomised trials? 1 (2%) 16. Does involvement of (i) members of the public and/or (ii) members of the public participating in a randomised trial, in designing trial information improve recruitment?

(6%)
17. What are the incentives that motivate members of the public to agree to participate in randomised trials?

(10%)
18. Do trial recruiters who have received focused, specialised recruitment training achieve better levels of recruitment than non-trained recruiters? 2 (4%) 19. Do randomised trials which have a low participant burden/requirements achieve better recruitment than those that have a greater patient burden? 0 (0%) 20. Does a central registry for members of the public (i.e. a list of people with contact details) who are interested in taking part in a randomised trial improve recruitment?
The recruitment SWAT did not map onto a PRioRiTy 1 top 20 question 5 (10%) Some SWATs mapped onto more than one PRioRiTy question leading the sum of percentages to be more than 100%.
2021/06/swat_network/), which provide guidance and advice on all aspects of SWAT design, implementation, analysis and reporting. A list of priority replications has been provided by Treweek et al. (2018 15 ) and Gillies et al. (2021 16 ) Cochrane systematic reviews in addition to this SWAT mapping analysis.

Conclusion
There is a mismatch between SWAT activity and prioritised unanswered research questions in recruitment and retention coming from PRioRiTY 1 and 2. Many questions considered research priorities by trial stakeholders have no or very little SWAT activity, while others have many, slightly different intervention evaluations from which it is, or will be, hard to make confident judgements about effects. Trial teams should consider the PRioRiTy 1 and 2 priorities for recruitment and retention, respectively, when designing their SWATs. Moreover, in many cases replication is likely to make a greater contribution to the evidence base than further innovation. 19. What are the best strategies for using participant incentives (e.g. monetary or nonmonetary) and how should they be implemented (e.g. when should they be provided) when collecting information from participants in clinical trials?
10 (28%) 20. How does continuity (e.g. seeing/speaking to the same staff) and consistency (e.g. of trial information) affect retention? 0 (0%) Some SWATs mapped onto more than one PRioRiTy question leading the sum of percentages to be more than 100%.

Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Trials Methodology Research Partnership (MR/S014357/1). The Health Services Research Unit, Institute of Applied Health Sciences (University of Aberdeen), is core-funded by the Chief Scientist Office of the Scottish Government Health and Social Care Directorates. The funders had no involvement in study design, collection, analysis and interpretation of data, reporting or the decision to publish.

Supplemental Material
Supplement material for this article is available in online.