Is two better than one? Maximising follow-up of self-reported outcome measures for a stroke survivor population: Results from a study within a trial in the LoTS2Care feasibility study

Background: Improving outcome data collection rates is an essential part of managing clinical trials and ensures statistical power and generalisability of results are maintained. Studies within a trial (SWATs) provide a robust methodology to investigate the most efficient methods to maximise outcome follow-up. Methods: LoTS2Care, a feasibility cluster Randomised Controlled Trial, recruited 269 stroke survivors across 10 services, and incorporated a SWAT to evaluate the effect of questionnaire booklet format (one booklet or two) on follow-up rates for self-reported postal outcomes at 6 and 9 months post-recruitment. Available participants were individually randomised (1 : 1) by the Clinical Trials Research Unit and follow-up rates in the two groups were compared. Results: At 6 months post-recruitment, 254 participants were randomised: 126 to receive (125 posted) the single booklet; 128 to receive two booklets. By 9 months post-recruitment, 116 and 123 participants were still available in each group, respectively. For participants randomised to two booklets, return of at least one of the booklets was considered as ‘followed-up’. At 6 months, 114/125 (91.2%) participants sent the single booklet returned it, compared to 108/128 (84.4%) sent two (odds ratio (OR) 1.92, 95% confidence interval (CI) 0.88 to 4.19). By 9 months, 108/116 (93.1%) participants returned the single booklet, compared to 105/123 (85.4%) sent two (OR 2.31, 95% CI 0.97 to 5.55). Conclusions: The SWAT was an inexpensive, straightforward way to test how booklet format affected follow-up rates. Larger participant numbers would be required for conclusive results. These initial findings, however, suggest that including all outcome measures in a single booklet may maximise return rates, especially in trials with similar populations, such as those living with brain injury, cognitive or speech impairment or older people.


Background
Whilst much focus is placed on participant recruitment in the management of clinical trials, maximising the collection of outcome data is essential in ensuring that statistical power and generalisability of results is maintained. 1 A Cochrane review of 38 clinical trials which implemented 'randomised or quasi-randomised evaluations of strategies to increase retention' identified questionnaire format as one of six broad types of strategies to improve retention. 2 LoTS2Care, a feasibility cluster Randomised Controlled Trial (cRCT), part of an National Institute for Health Research programme grant for applied research, recruited 269 stroke survivors across 10 stroke services, along with 85 carers, at approximately 6 months post-stroke. 3,4 Participants were followed-up at 3 months, 6 months and 9 months post-registration date using postal questionnaires to collect self-reported outcomes.
Of the 269 registered participants, the mean (SD) in age (years) was 73 (11). The mean (SD) NIH Stroke Scale (NIHSS) score was 4.8 (5.12); although missing data for this score was high. The level of language ability post-stroke varied with 87 recorded as having normal ability, six participants as having dysphasia, 3 recorded as having dysarthria, 170 recorded as 'not known' and 3 as missing. Fuller breakdowns of trial population characteristics can be viewed in Tables 13 and 14 of Forster et al. 2021. 4 The booklets comprised several measures, including all primary and secondary self-report outcomes and the health economics questionnaire. To investigate how questionnaire format might impact questionnaire follow-up rates, the LoTS2Care team designed a 'Study Within a Trial' (SWAT). The SWAT investigated the use of two different booklet formats. The standard booklet (19 pages) combined all outcome measures and health economic questionnaires into a single but lengthy booklet. The other divided the measures into two shorter booklets (12 and 7 pages each) to be implemented at the follow up points at 6 and 9 months post-recruitment. There was an assumption that shorter booklets may be easier to complete in case the participant became fatigued. By splitting the booklets into two, it was thought that participants might be encouraged to take a break and the return to the second booklet. It was also suggested that participants experiencing questionnaire fatigue may be inclined to return at least one booklet and therefore maximising data collection of the key clinical outcomes in this instance. Thus, the objective of the SWAT was to answer the following question: Does the administration of two shorter questionnaire booklets increase postal follow-up rates compared to one longer questionnaire booklet in longer-term stroke survivors?
The SWAT would provide evidence to answer this key question in order to best inform the method for collection of patient reported outcome data in the LoTS2Care definitive trial.
The SWAT adheres to most of the 'key features of a typical SWAT', as highlighted in the Trial Forge Guidance, 5 with the exception of not having a formal protocol; the SWAT was briefly described in the main trial protocol. 3 It addresses several of the top 10 research questions raised in the PRioRiTy II study, such as how to minimise burden on staff and participants, and how to encourage participants to complete tasks by experimenting with mode of delivery of the questionnaire, and assessing how this affects the data returned. 6

Methods
Host trial: LoTS2Care feasibility trial design LoTS2Care evaluated the 'New Start' intervention (with usual care) (NS) versus Usual Care (UC) alone, delivered at 6 months post-stroke and designed to improve quality of life by addressing unmet needs and enhancing participation in stroke survivors. Ten stroke services were allocated to either NS or UC at a 1:1 ratio by minimisation with a random element, using size of stroke service (>300 patients per annum; ≤300 patients per annum) and whether or not the stroke service currently had a 6-month post stroke review service in place, as binary minimisation factors. LoTS2Care was a feasibility study and therefore a formal power calculation was not required as effectiveness was not being assessed.
Participants were recruited by post and registered onto the trial at approximately 6 months post-stroke. Available carers also were recruited by post. Postal recruitment methods may have primed participants to receive and complete questionnaires by post, explaining the high overall follow-up rates achieved in the trial (80.3% at 9 months). Participants within the LoTS2Care trial completed a 28page baseline booklet at registration. Baseline data was either completed in person or by post and if requested, with researcher assistance. All follow up data (at 3, 6 and 9 months) was collected via self-reported postal questionnaires. At 3 months all participants were sent a 16-page booklet.

Study within a trial design
At 6 months post participant registration, a two arm SWAT was undertaken. Immediately prior to initiation of the 6 month follow-up point, participants were individually randomised (1:1) to receive either the single standard 19page booklet, or two separate booklets, of 12 and 7 pages. The allocated booklet format was posted to the stroke survivor at 6 and 9 month follow-up points. Participants were allocated via stratified randomisation with fixed blocks of size 4, using the CTRU Gen-24 randomisation system. The two stratification variables were host trial centre allocation (control or intervention) and mode of administration of baseline assessments (postal, face to face or unknown).
Where applicable, the carer was posted the format allocated to the stroke survivor.
The primary objective of the SWAT was to determine whether two shorter questionnaire booklets increased postal follow-up rates compared to one longer questionnaire booklet in longer-term stroke survivors. Booklets that were returned blank were considered as 'not followed-up', although partially completed booklets were considered as 'followed-up'. Participants randomised to two booklets were considered followed-up if they returned at least one non-blank booklet. Item completion rates were also assessed as part of measurement of follow-up rates. All participants who were individually randomised to the SWAT and available (i.e. not died, withdrawn, or lost to follow up) at the respective mailing time points were included in the analysis populations. Logistic regression was used for the analysis and restricted to between-group estimates of follow-up presented as odds ratios with 95% confidence intervals.
As LoTS2Care was a feasibility trial, no formal sample size calculation was performed, nor was one performed for the SWAT, which is in line with SWAT methodology. 5,7 The LoTS2Care feasibility trial protocol was previously published and includes a brief description of the SWAT design. 3

Results
The LoTS2Care trial recruited a total of 269 participants. Of these, 254 participants were available (i.e. not died, withdrawn, or lost to follow up) at 6 months postrecruitment for randomisation into the SWAT. 126 participants were randomised to receive a single booklet and 128 participants were randomised to receive two shorter booklets; one of the participants randomised to receive a single booklet withdrew shortly after SWAT randomisation, hence 253 participants in total were sent booklets. By 9 months post-recruitment, 239 participants were available and included in the SWAT analysis population. Figure 1 demonstrates the flow of patients through the SWAT, including the follow-up rates and total figures. For summaries of this study within a trial, only booklets returned via post are considered. Outcomes completed via phone are not used in summarising completion rates but are included in the main study results The baseline characteristics of both SWAT groups were compared to check that the groups were representative of the trial (and wider stroke survivor population) as a whole. Table 1 shows figures and percentages for key baseline characteristics of the two SWAT groups compared to the total SWAT population and total trial population. Although those randomised to two booklets were slightly more likely to be male than the trial on average, overall the SWAT population was deemed to be representative and the two SWAT groups well balanced.
Overall, across all participants and over all time-points, follow-up rates were high. At 6 months, participants were more likely to return follow-up questionnaires when administered together as one single booklet than as two shorter booklets, with 114 out of 125 (91.2%) participants sent the single booklet returning it, compared to 108 out of 128 (84.4%) sent two booklets (OR 1.92, 95% confidence interval (CI) 0.88 to 4.19). Similar trends were observed at 9 months, with 108 (93.1%) of the 116 participants in the one booklet group returning their booklets, compared with 105 out of 123 (85.4%) in the two booklet group returning both booklets (two (1.6%) returned a single booklet containing the outcome measures only), corresponding to an odds ratio (OR) of completing one single booklet compared to completing two booklets of 2.31 (95% confidence interval 0.97 to 5.55).
For those returning questionnaires at 9 months, itemcompletion rates were also affected by questionnaire format. The Health Economics questionnaire alone and the full battery of measures (i.e. all health outcomes and resource use) were more likely to be completed in the single booklet arm compared to the two booklets arm. Completion rates of health-care resource use questionnaire alone were 98.1% vs. 89.7% of those returning the booklet (91.4% vs. 78.0% of those sent); both full outcomes completion rates were 97.2% vs. 87.9% respectively (90.5% vs. 76.4% of sent), Table 2.
Carers were also more likely to return the questionnaires when sent a single booklet. At 9 months, 30/34 returned the single booklet (88.2% of sent); whereas 30/37 (81.1% of sent) returned both. For those returning questionnaires, there was only a very small difference between groups at 6 months and there were no differences in item-completion rates between groups at 9 months. When, however, assessed as a proportion of those sent questionnaires, all types of outcomes at both time points were more likely to be completed for those carers sent a single booklet. The number of carers available and assessed in this analysis is low; thus, this finding should be interpreted more cautiously than that for stroke survivors, see Table 3.
Overall, these results suggest outcome follow-up rates are maximised when sending all required questionnaires within a single, longer booklet, rather than splitting across two, shorter booklets.

Discussion
The results of the SWAT show increased odds of stroke survivors returning questionnaires when sent one booklet, as opposed to two booklets. Numbers within the SWAT were small and therefore these results did not reach statistical significance. This finding is contrary to that reported in a 2009 Cochrane review looking at methods to increase response rates in postal and online questionnaires, 8 which found that shorter questionnaires had increased odds of being returned. However, this comparison does not take into account the fact that we had two shorter booklets, which together contained the same measures (and therefore length) as the single booklet.
Carers were also more likely to return the longer, single booklet, although the result should be interpreted with caution due to low number of available carers assessed.
It is also of note that the questionnaire return rates for the study were already high, leaving limited room for improvement.
The study involved a small sample, limiting generalisability of the results; a much larger sample would be required for conclusive findings. Moreover, the study could have benefitted from recording participant-reported preference for booklet format. Nonetheless, the format of the booklets was developed and tested in a target population and the results inform assessment in further stroke trials in order to maximise follow-up rates.
Most SWATs are implemented in full-scale trials and often attempt to answer broader questions in order to improve the evidence base for trial conduct. Only one SWAT on the SWAT-SWAR repository is listed currently as a feasibility trial SWAT.
For example, one current trend emphasises use of standard SWATs in a larger number of host trials to increase generalisability of SWAT results and maximise the impact of SWAT research. Although these steps are essential in informing best and most effective practice in running large-scale trials, it is also fruitful to explore implementation of SWATs in smaller, more specific trial contexts. SWATs can be used as a low-cost way to test trial processes and conduct for a very specific population, disease area, or type of trial. Feasibility studies (provided they are of reasonable size) make natural host trials for SWATS. Small-scale SWATs implemented in feasibility trials can be an efficient way to test aspects of trial conduct in order to optimise methods for the large-scale trial. We would therefore urge trialists to explore ways to test key trial processes at feasibility stage in order to maximise trial efficiency and ensure success in definitive trial processes.

Conclusions
The SWAT was a simple and straightforward way to test how booklet length and number of booklets sent at one time affected follow-up rates in an older, stroke survivor population. Whilst the findings are not conclusive, they do suggest that in this particular setting for this population, including all self-reported questionnaires in one booklet was  most likely to maximise follow-up rates. These initial findings may also be relevant in trials with similar populations, such as those living with brain injury, cognitive or speech impairment or older people. Study within a trials embedded in feasibility trials have the ability to inform the best methods for trial conduct in a specific trial target population and can therefore offer a unique opportunity to maximise definitive trial efficiency.