Site staff perspectives on communicating trial results to participants: Cost and feasibility results from the Show RESPECT cluster randomised, factorial, mixed-methods trial

Background/Aims Sharing trial results with participants is an ethical imperative but often does not happen. Show RESPECT (ISRCTN96189403) tested ways of sharing results with participants in an ovarian cancer trial (ISRCTN10356387). Sharing results via a printed summary improved patient satisfaction. Little is known about staff experience and the costs of communicating results with participants. We report the costs of communication approaches used in Show RESPECT and the views of site staff on these approaches. Methods We allocated 43 hospitals (sites) to share results with trial participants through one of eight intervention combinations (2 × 2 × 2 factorial; enhanced versus basic webpage, printed summary versus no printed summary, email list invitation versus no invitation). Questionnaires elicited data from staff involved in sharing results. Open- and closed-ended questions covered resources used to share results and site staff perspectives on the approaches used. Semi-structured interviews were conducted. Interview and free-text data were analysed thematically. The mean additional site costs per participant from each intervention were estimated jointly as main effects by linear regression. Results We received questionnaires from 68 staff from 41 sites and interviewed 11 site staff. Sites allocated to the printed summary had mean total site costs of sharing results £13.71/patient higher (95% confidence interval (CI): −3.19, 30.60; p = 0.108) than sites allocated no printed summary. Sites allocated to the enhanced webpage had mean total site costs £1.91/patient higher (95% CI: −14, 18.74; p = 0.819) than sites allocated to the basic webpage. Sites allocated to the email list had costs £2.87/patient lower (95% CI: −19.70, 13.95; p = 0.731) than sites allocated to no email list. Most of these costs were staff time for mailing information and handling patients’ queries. Most site staff reported no concerns about how they had shared results (88%) and no challenges (76%). Most (83%) found it easy to answer queries from patients about the results and thought the way they were allocated to share results with participants would be an acceptable standard approach (76%), with 79% saying they would follow the same approach for future trials. There were no significant effects of the randomised interventions on these outcomes. Site staff emphasised the importance of preparing patients to receive the results, including giving opt-in/opt-out options, and the need to offer further support, particularly if the results could confuse or distress some patients. Conclusions Adding a printed summary to a webpage (which significantly improved participant satisfaction) may increase costs to sites by ~£14/patient, which is modest in relation to the cost of trials. The Show RESPECT communication interventions were feasible to implement. This information could help future trials ensure they have sufficient resources to share results with participants.


Background/aims
Offering trial participants the results of trials they have participated in is an important part of the ethical conduct of clinical trials. 1,24][5][6][7] However, in practice, it often does not happen or is done poorly.The 2021 UK Health Research Authority research transparency report states that '90% of clinical trials have not told participants about findings'. 8A 2016 survey of authors of clinical trials publications from 2014 to 2015 found that only 27% of them reported disseminating results to participants. 9arriers to communicating results include practical challenges; concern about the impact of sharing results; uncertainty about how to do it; lack of guidance or incentives 9,10 or researchers simply not thinking about it. 113][14][15][16] Sharing of results often occurs after the trial funding period, making it difficult to cover these costs.][17] The literature on the feasibility and resource requirements for different approaches to sharing results with trial participants is sparse.The US Center for Information and Study on Clinical Research Participation reported that, when results were returned, each site spent around 30 min to 2 h sending out result summaries to trial participants 18 ; although the number of participants at these sites was not discussed.Another study reported the costs of £1624 for running an online meeting for participants and other stakeholders. 19However, this cost estimate does not include the 40 h of staff time required to organise the event.The other cost estimate reported in the literature is from a study which sent out leaflets by post, with printing and postage cost coming to £1.22 per participant, excluding the leaflet development and mailing time. 5Better information on what are the resource requirements for approaches to sharing results could help researchers plan appropriately.
The Show RESPECT study elucidated how to share trial results with participants, testing three approaches in a cluster randomised factorial trial, 7,20 within the context of the ICON8 ovarian cancer trial. 21Data collected from trial participants showed that mailing printed summaries, alongside a link to a webpage 6 email list, resulted in higher satisfaction with how the results were shared than the link to a webpage 6 email list without the printed summary. 7,20Full results from the Show RESPECT study have been published as part of a doctoral thesis. 20This article aims to explore the resource implications and views of site staff on the acceptability and feasibility of these approaches to sharing results.

Methods
We used a mixed-methods approach to collect data from site staff on the resources required for sharing results (reported at cluster (site) level) and their views on the process, concerns and challenges faced (reported at individual level).The qualitative and quantitative data have equal weight in their contribution to addressing the research aims.For clarity, we refer to ICON8 participants as 'patients' and site staff who contributed data to Show RESPECT as 'site staff'.The study protocol is available online. 22

Supplemental Table S1 contains the Consolidated Standards of Reporting Trials (CONSORT) 2010 checklist.
Trial design.Show RESPECT was a cluster randomised 2 3 2 3 2 factorial trial within the ICON8 ovarian cancer trial (ISRCTN10356387). 21We randomised each UK trial site (secondary or tertiary hospital) (cluster) in ICON8 that agreed to take part in Show RESPECT to a combination of interventions to communicate ICON8 results to participants. 7Sites were stratified for randomisation by the number of ICON8 patients alive at the time the results were available (small \ 6, medium 6-11 and large ø 12).The primary outcome for the study overall, reported previously, was patient satisfaction with how the results were shared. 7,20This article reports results around site staff perspectives on communicating results to participants.
Interventions.Staff at participating sites mailed a Patient Update Information Sheet to all ICON8 patients who were known to be still alive informing them that trial results were available together with access information, including the URL of their randomised webpage (basic or enhanced) and email list sign-up instructions if randomised to the email list.The Patient Update Information Sheet told patients at sites randomised to the printed summary that they would be mailed a printed summary of the results after 3 weeks and how to opt out of receiving it.A full description of the interventions is contained in our previous report. 7,20rticipants.For the results presented in this report, participants were site staff who had been involved in sharing the ICON8 results with people taking part in ICON8 (e.g. through mailing information or answering queries).
Outcomes.The primary outcome for this sub-study of Show RESPECT was the cost to site per participant.This is a composite endpoint made up of An estimate of the cost of the time taken to deliver the interventions at a site (multiplying the reported time taken by the cost per hour for the job role of the staff member(s) who delivered the interventions taken from the National Institute of Health Research Schedule of Events Cost Attribution Template, assuming a medical research charity funder); An estimate of the cost of the time taken to deal with queries; Any non-staff costs associated with result dissemination incurred by sites.These costs were divided by the number of ICON8 patients at the site who were alive at the time results were shared.
Secondary outcomes from site staff data are listed in the S2 Text of the Supplemental Material.
Quantitative data were elicited from site staff by one questionnaire completed immediately after sharing results and another completed 2-3 months later (S3 Text of the Supplemental Material).Site staff involved in sending out printed information to patients were asked to complete the first questionnaire, and site staff involved in dealing with patient queries were asked to complete the second questionnaire.For some sites, this meant both questionnaires were completed by the same people, and for other sites, different people completed the two questionnaires.Data were collected from December 2018 to September 2019.All site staff involved in sharing results were asked to complete questionnaires while recording the time they (individually) spent on this.Where more than one member of staff returned questionnaires, the time (and costs, estimated based on their job role) was added together to estimate the total time/cost for that site.
The trial ended 4 months after the last batch of clusters was randomised, as further data were unlikely.
Sample size.Show RESPECT was powered based on the primary outcome (patient satisfaction). 7No sample size calculations were carried out to assess power for outcomes collected from site staff.
Randomisation.Sites in Show RESPECT were cluster randomised using a factorial approach to a combination of enhanced versus basic webpage; printed summary versus no printed summary; email list invitation versus no invitation.Sites were randomised in blocks of eight through random permutations within blocks.Full details of our randomisation approach can be found in our previous report. 7inding.It was not feasible to blind site staff to their site's intervention allocation.
Statistical analysis.This analysis was conducted under the intention to treat (ITT) principle.We analysed data from site staff data according to the interventions their cluster was allocated to.The prior assumption in the trial design was that there would not be any important interactions between the webpages, printed summary and email list.Therefore, our analysis is of the main effects of each intervention, adjusting for the others in the regression models (alpha = 0.05).To reflect the way in which randomisation was carried out, we adjusted for site size stratum by including this as a variable in the regression models.We did not impute any missing outcomes, as we had no information to inform our imputation.
We estimated effect measures for the interventions based on regression models, using linear regression to analyse the costs of the interventions, ordinal logistic regression for ordinal outcomes and logistic regression for binary outcomes.Models include random effects for site.We used mean differences to summarise the effects for continuous outcome measures and odds ratios for binary and ordinal outcomes.As the population for this analysis is small, no subgroup analyses were performed.
Statistical analyses were performed using Stata version 16.1 (StataCorp LLC, College Station, TX).

Qualitative methods
Supplemental Table S4 shows the Consolidated Criteria for Reporting Qualitative Studies (COREQ) checklist.
Qualitative sampling.Invitations to take part in interviews were sent by email to site staff involved in Show RESPECT.Purposive sampling included staff from sites offered the range of Show RESPECT interventions and different job roles (nursing, oncologist and administrative roles).
Qualitative data collection.Semi-structured interviews with site staff were carried out either face-to-face (onsite) or by telephone by A.S. S5 Text of the Supplemental Material contains more details about the research team and reflexivity.Semi-structured interviews were informed by a topic guide (S6 Text of the Supplemental Material), which was revised as interviews proceeded to explore emergent issues. 23Only the interviewee and interviewer were present during interviews.The interviews were audio recorded and transcribed verbatim.Field notes were made immediately afterwards.Transcripts were checked against the recordings for accuracy, and identifying data were redacted.Transcripts were not returned to interviewees for checking.Repeat interviews were not conducted.Free-text questions within the questionnaires invited comments alongside the quantitative data.
Qualitative analysis.A thematic analysis 24 of qualitative data, taking a critical realist stance, was conducted by A.S. in Atlas.tiversion 8.4 (ATLAS.tiScientific Software Development GmbH), as detailed previously. 7Inductive thematic saturation was reached at the 11 th interview, as was data saturation.Meetings were held with site staff once the initial analysis had been carried out to check our interpretation and messages (rather than for data collection).A 'following the thread' approach was used to triangulate the results of the qualitative and quantitative components of the study at the analysis stage: The initial analysis of the two types of data was carried out to identify key themes and questions, then the other type of data was examined to see what light it could shed on those themes and questions. 25

Ethics
Show RESPECT obtained ethics approval from the London-Chelsea Research Ethics Committee, MREC number 18/LO/1011.Both questionnaires contained an embedded informed consent element, in line with the UK Health Research Authority's guidance on proportionate approaches to informed consent for self-administered questionnaire-based research, 26 with completion and return of the questionnaire taken to indicate consent to use the data has been given.Qualitative interviewees gave written informed consent before the interview started.

Participation in Show RESPECT
Show RESPECT randomised 43 sites.We received at least one questionnaire from 68 staff members from 41 sites.No questionnaires were returned from two randomised sites due to staff turnover.Figure 1 shows the CONSORT diagram, including the number of sites where no responses were received for the two site staff questionnaires by each of the eight combinations of interventions.Table 1 shows the number of sites and questionnaires received by the three factorial randomisations.
Most respondents were in nursing (63%, 42) or administrative (30%, 20) roles.Most respondents worked on multiple trials simultaneously, with 72% (49) working on more than 10 trials at the time they completed the questionnaire.Around half (33) had been working on ICON8 for 2 years or less, while 38% (26) had worked on ICON8 for more than 5 years.Table 2 shows site staff respondent characteristics.
We interviewed 11 site staff from 12 sites (mean duration 52 min, range: 35-75 min): two from small strata sites, five from medium strata sites and five from large strata sites.Most interviewees were in nursing (5)  or administrative (4) roles, while two were oncologists.Table 3 shows the characteristics of interviewees.We recruited at least the target number of interviewees in all parts of the sampling frame.

Resources required from sites for sharing results with patients
The resources required from sites to share the results with patients include staff time for posting information and dealing with patient queries and the costs of postage and stationery (printed copies of materials were provided by the clinical trials unit).Table 4 shows the estimated total costs per patient and a breakdown by the different types of resources required.Total costs were £13.71/ patient higher (95% confidence interval (CI): 23.19, 30.60) in the printed summary group than that in the no printed summary group (where only the Patient Update Information Sheet, informing patients how to access the webpage 6 the email list, was mailed), but this difference was not statistically significant (p = 0.108).The differences in costs of the other comparisons were smaller and also not statistically significant.There was no evidence of interaction between interventions.Supplemental Table S7 shows the average costs to site per patient by each of the eight factorial combinations.The biggest component of the total costs was staff time  9) 0 (0) 2 (10) 0 (0) 0 (0) (9) Small sites 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) (0) Medium sites 1 (8) 1 ( 14) 0 (0) 1 ( 17) 0 (0) 0 (0) (14) Large sites 1 (8) 1 ( 14) 0 (0) 1 ( 14) a Sites were classified as small if they had fewer than 6 ICON8 participants alive at the time results were available, medium if they had 6-11 participants and large if they had 12 or more participants.b Number of sites for which we had no responses (at least one response was expected from each site).c Number of sites for which we had no responses, who had not confirmed that they received no queries from participants (CRF3 was only expected for sites that had received queries from participants). to send out the printed information, taking, on average,

Site staff views on the experience of sharing trial results with patients in Show RESPECT
Overall views of the process of sharing results in Show RESPECT.Site staff described their experiences of sharing results in positive terms, as 'easy', 'not complex',  'achievable', 'not time-consuming' and 'working well'.They appreciated the 'clear instructions' provided.
Preparing patients to receive the trial results.The need to prepare patients to receive results was discussed repeatedly in site staff interviews and questionnaires.The importance of preparing patients stemmed partly from recognition that not all would want to know the results, and so it was important to offer the opportunity to opt in or out.Views differed on the timepoint at which this should be done, with some suggesting it should be done when patients join the trial.Others thought patients might change their minds between joining the trial and the results becoming available and so would need to be asked nearer the time: I don't believe in just sending information without first asking for their consent or asking them if they're interested in doing this, because some do want to know and some don't want to know.(BMRNI04: research burse, medium strata site) Another driver behind the need to prepare patients prior to sharing results was that results may potentially be upsetting or confusing: If the results are difficult to interpret or complex you may want to add something like the results are complex and you may want to go through it with your doctor, or something like that.(EBLMCLI02: oncologist, large and medium strata sites) In Show RESPECT, the Patient Update Information Sheet was designed to prepare patients to receive results and to give them the opportunity to opt-out or access the interventions.Some site staff added an additional preliminary step, telephoning or talking to patients in clinic to inform them that the Patient Update Information Sheet was coming: We sent the information sheets out, then contacted them and let them know they're coming ... That's a really good way of doing it because if that had just ended up on their doorstep, they'd have read it and then just probably thrown it in the bin because it didn't come with any compliment[s] slip or they didn't really know what it was.(GMTCI02: trial coordinator, medium strata site) However, not all site staff agreed that this step was needed or that it was a good idea: I don't agree with phoning the patients, just because, you know, a lot of our patients, you know, are busy with dayto-day life and it's not, I don't think it's nice just calling them and reminding them of it all.(FLTCI01: trial coordinator, large strata site) Further follow-up and support.There was recognition that some patients may need further support or have questions about the results.Some site staff phoned patients after they sent the Patient Update Information Sheet or printed summary to see if they had any questions or needed further support.Others included a note with the results inviting patients to make contact with any questions.
Concerns and challenges.Most site staff questionnaire respondents (88%, 60/68) reported having no concerns about how they shared the ICON8 results with patients, with no statistically significant differences between the randomised interventions.Similarly, around three quarters of site staff reported not finding anything challenging about sharing the ICON8 results, again with no statistically significant differences between the arms (Table 5).Some of the site staff were concerned about the emotional impact of sharing results: Some were uncomfortable sharing information on average progression-free survival times, while others were concerned that those who felt they had benefitted from trial participation may be upset to learn that the trial did not find a benefit overall.Oncologists expressed the need for care in how sharing results is done to ensure patients are not unnecessarily distressed or confused.Patient and public involvement in the preparation of information about results for patients was identified as a way of mitigating the risk of causing upset: We do have a duty to give the patient information, it's just being wise and careful to give that information well, in a way that patients can understand ...And like any information we have to time that well and be sensible about who we're giving that to.(EBLMCLI02: oncologist, large and medium strata sites) The other main challenges identified by site staff were the time needed to share results via mailed information.This was particularly an issue for staff at sites with larger numbers of ICON8 patients.Staff at smaller sites recognised that although it may not have been a problem for them in this study, it would be challenging in studies with many patients at a site: It was just so time-consuming to send, you know, the results out to so many patients.(HLRNI03: research nurse, large strata site) If site staff had not had contact with a patient for a while and were aware the patient was unwell, this did raise some questions over whether they should send the results.Similarly, if patients had transferred between sites, it could cause confusion over who was responsible for sharing results with that patient.For some site staff, giving information out remotely, rather than faceto-face, was challenging as they could not gauge the reaction of patients.
Dealing with queries from patients.Just over a quarter of site staff questionnaire respondents reported that no patients contacted them with queries, while almost 60% were only contacted by one or two patients (Table 5).There were no significant differences between the randomised arms in the number of patient queries.Eightythree percent of questionnaire respondents felt it was quite or very easy to deal with queries patients raised, with no significant between-arm differences.
Sharing results in future trials.Most site staff respondents said the way they had shared results with patients in Show RESPECT should be the standard approach for other trials (76%), with no significant differences between the randomised arms.Similarly, 79% said they would not do anything different for future trials (Table 5).

Discussion
Providing results to patients in the form of opt-out printed summaries increased costs to sites of sharing results by around £14 per patient.Most of that increased cost was attributable to staff time in mailing printed information.These costs, and those incurred by the clinical trials unit, are small (1% or less) compared with the overall cost of trials, which previous research has estimated to be around £2987 27 or £7890 28 per participant on average, in the UK.
The processes and methods used in Show RESPECT to share results with patients were seen as both appropriate and feasible by site staff.Preparing patients to receive trial results was an important step in this process (similar to the findings of the BRACELET study). 29ome site staff viewed the Patient Update Information Sheet alone as sufficient, while others felt more comfortable talking to the patients first to let them know what to expect, and some felt it should have been covered in the informed consent process when patients joined the trial.Mailing out the printed information was generally reported to be easy and not too time-consuming although staff at sites with large numbers of ICON8 patient participants found it more burdensome.Site staff received few queries from patients about the results.
A key strength of these results is that they are from a mixed-methods randomised controlled study within a trial.The randomised design allows us to be confident that the differences we observed were due to the interventions rather than other factors, while the qualitative data allow us to explore the views of site staff on their experience of sharing trial results within the Show RESPECT study.The study included sites of different types and numbers of ICON8 patient participants, allowing us to transfer our results on feasibility and acceptability to other trials with similar site characteristics.However, Show RESPECT only looked at sharing the results of a single trial, and care needs to be taken when transferring the results to other trials in different diseases, health systems and results scenarios.Show RESPECT was not powered to detect differences in secondary outcome measures (which includes the outcomes reported in this article).This means there may have been real differences we were unable to detect.The questionnaires used to collect data on time spent on activities asked respondents to select the category that reflected the time they had spent, rather than giving precise figures.Similarly, our costs of time for different job roles at sites are based on a generic research costing tool, rather than the actual cost of each individual staff member.This means our cost figures are estimates rather than precise reflections of the time and cost of these activities.The resources required at the clinical trials unit level, reported in the S8 Text of the Supplemental Material, do not include the costs of translation, which should be included in future studies to ensure people who have different first languages are able to access the results. 30Another limitation is our cost estimates do not distinguish between fixed and per-patient costs.Economies of scale mean per-patient costs to the clinical trials units are likely to be lower in trials with more patients.There may also be economies of scale at the site level, meaning average costs to site per patient may be lower at larger sites and in larger trials.
Previously reported results from Show RESPECT showed that printed summaries improved patient satisfaction with how the results were shared and enabled more people who wanted to know the results to find them out. 7The results reported here show that this approach is feasible for site staff to implement and acceptable to them.This echoes findings from previous research, where study staff reported that disseminating trial results summaries was simple, straightforward and not time-consuming and that patient queries were not common and did not require substantial amounts of time to deal with. 31e believe this is the first study to provide detailed information about the costs of different approaches to sharing results with trial participants from both a site and clinical trials unit perspective.Our results include the human resource, printing and posting costs and therefore reflect a more accurate cost estimate.These costs should inform those planning results dissemination in future trials to ensure adequate resources are available (both budget and staff time) to share results with patients.Funders have a role to play in encouraging researchers to do this and in ensuring plans for sharing results are implemented.Further research is needed to determine the extent to which approaches to sharing results with trial participants need to differ for different populations, disease states, study questions and settings.

Conclusions
We found that the process and communication approaches used within the Show RESPECT study for sharing results with patients were acceptable and feasible for site staff.The combination of a webpage and printed summary, which resulted in the highest participant satisfaction with how the results were shared, came at a modest cost that could be incorporated into trial budgets.The information on the process and resource requirements for the approaches used in Show RESPECT can guide others seeking to plan for sharing results.

Table 1 .
Recruitment of sites and site staff respondents.

Table 3 .
Site staff who were interviewed for the qualitative study.
a One interviewee worked at two sites of different sizes, which were randomised to different interventions.

Table 4 .
Resources used by sites to share results with patient participants, per participant.
CI: confidence interval.a Adjusted for strata and other randomisations.b Primary outcome for site staff data.