A narrative review of current evidence supporting the implementation of electronic patient-reported outcome measures in the management of chronic diseases

An application of telemedicine of growing interest and relevance is the use of personal computers and mobile devices to collect patient-reported outcomes (PROs). PROs are self-reports of patients’ health status without interpretation by anyone else. The tools developed to assess PROs are known as patient-reported outcomes measures (PROMs). The technological innovations that have led to an increased ownership of electronic devices have also facilitated the development of electronic PROMs (ePROMs). ePROMs are a conduit for telemedicine in the care of patients with chronic diseases. Various studies have demonstrated that the use of ePROMs in routine clinical practice is both acceptable and feasible with patients increasingly expressing a preference for an electronic mode of administration. There is increasing evidence that the use of electronic patient-reported outcome (ePROMs) could have significant impacts on outcomes valued by patients, healthcare providers and researchers. Whilst the development and implementation of these systems may be initially costly and resource-intensive, patient preferences and existing evidence to support their implementation suggests the need for continued research prioritisation in this area. This narrative review summarises and discusses evidence of the impact of ePROMs on clinical parameters and outcomes relevant to chronic diseases. We also explore recently published literature regarding issues that may influence the robust implementation of ePROMs for routine clinical practice.


Introduction
Telemedicine refers to the use of electronic information and communications technology to provide and support healthcare remotely. 1,2 An application of telemedicine of growing interest and relevance is the use of personal computers and mobile devices to collect patient-reported outcomes (PROs). 3 PROs are self-reports of patients' health status without interpretation by anyone else. 4 The tools developed to assess PROs are known as patient-reported outcomes measures (PROMs).
Although the use of PROMs in clinical trials to assess the impact of interventions is well established, interest in their use in routine clinical practice is on the increase. The technological innovations that led to an increased ownership of electronic devices have also facilitated the development of electronic PROMs (ePROMs). 5 PROMs are traditionally paper-based while the term 'ePROMs' refers to telephone-based interactive voice response systems and screen-based systems. 6 ePROMs may be administered at clinic, with or without clinical supervision, or remotely in an unsupervised setting (such as subject's home or workplace). Remote assessment enables more timely and accurate self-reporting and may avoid potential recall bias. 6 The National Centre for Chronic Disease Prevention and Health Promotion (NCCDPHP) defined chronic diseases as conditions that last a year or more and require ongoing medical attention or limit activities of daily living or both. 7 The definition encompasses conditions such as heart diseases, cancer, chronic lung disease, stroke, Alzheimer's disease, diabetes and chronic kidney disease (CKD). Most of these diseases are presently incurable. Patients with chronic diseases often require long-term care while experiencing significant symptom burden, which may impair their health-related quality of life (HR-QOL). 8 The use of ePROMs provides patients the opportunity to report remotely the impact of disease and treatment on their HR-QOL. Such information may inform the clinical management of patients. Various studies have demonstrated that the use of ePROMs in routine clinical practice is both acceptable and feasible, with patients increasingly expressing a preference for an electronic mode of administration. 9,10 However, despite these advancements, doubts about the quantifiable benefits of implementing PROMs/ePROMs for use in routine clinical practice persist and there are still concerns around integration with existing workflows. [11][12][13] Whilst the potential impact of PROMs/ePROMs on patientclinician communication has been well documented, [14][15][16] their impact on patient outcomes or clinical parameters has been limited. 17,18 A systematic review by Boyce and Browne found weak evidence that provision of PROM data to healthcare professionals resulted in a positive impact on patient outcomes. 19 The review also reported that studies that demonstrated the most significant impact used PROMs to facilitate patient care in an outpatient setting and there was weak evidence supporting their use as a screening tool. 19 The review included commentaries, editorials, systematic reviews and a limited number of primary studies.
The majority of the relevant primary studies available at the time of Boyce and Browne's review reported PROM projects in their nascent stages. [20][21][22][23][24] These studies were mostly crosssectional, feasibility or pilot studies, which are not designed to capture the impact of PROM data on patient outcomes. Furthermore, these studies were conducted prior to the publication of PRO-specific clinical trial guidelines such as the SPIRIT-PRO (Standard Protocol Items: Recommendations for Interventional Trials-PRO Extension) to facilitate the incorporation of PROs into clinical trial protocols and the CONSORT-PRO (Consolidated Standards of Reporting Trials-PRO Extension) to guide the reporting of PROM data from clinical trials. 25,26 Since the publication of the Boyce and Browne review in 2013, an increasing number of high quality, longitudinal studies aimed at measuring the impact of ePROMs on clinical outcomes have been reported. 27,28 This review aimed to summarise and discuss evidence of the impact of ePROMs on clinical outcomes relevant to the management of chronic diseases as defined by the NCCDPHP. We also explored recently published literature regarding issues that may influence the robust implementation of ePROMs for routine clinical practice. We focus on articles published since the Boyce and Browne 2013 review.

Search strategy
Prior to conducting this review, the authors were aware of the two highly cited articles by Basch et al. that reported compelling evidence of the impact of ePROMs on clinical outcomes since the publication of the Boyce and Brown review. 19,28,29 Forward citation searches of these three articles on the Web of Science (WoS) database (All Databases version) were conducted by OLA on 12 September 2020. 19,28,29 This approach facilitated the efficient retrieval of recent and relevant articles. In addition, Google Scholar was searched on 12 February 2021 to ensure that all relevant and recent articles were captured.

Screening process
Titles and abstracts were screened for relevance and full-text articles were obtained for articles that potentially met the eligibility criteria. journals.sagepub.com/home/taj 3

Eligibility criteria
Randomised controlled trials (RCTs), cohort studies and observational studies of any chronic disease were included if they reported the impact of ePROMs on clinical outcomes and/or healthcare resource use in chronic disease care. Specifically, our outcomes of interest included: patient survival, symptom management, treatment adherence and utilisation of healthcare resources. There were no language restrictions.
Articles were excluded if they reported: (i) studies relating to PROMs administered in paper form; (ii) clinician-reported instruments; (iii) pilot studies that only detailed the feasibility or acceptability of ePROMs or impacts on patient-clinician communications; and (iv); editorials, reviews, and conference abstracts.

Critical appraisal
The included studies were appraised using the appropriate critical appraisal skills programme (CASP) checklists. 30

Findings
The search on WoS retrieved 1099 entries for title and abstract screening. The full-texts for 52 were obtained for further review and 16 articles selected. Table 1 summarises the characteristics and findings of the included articles. Most of the articles reporting the impact of ePROMs on clinical parameters and outcomes in chronic conditions were derived from oncology. A few focussed on diseases such as rheumatoid arthritis, epilepsy and sleep apnoea. 27,31 All except one study were conducted in outpatient settings and most were RCTs. 32 There was evidence that the use of ePROMs in routine clinical practice may improve patient survival, symptom management and individualised care, treatment adherence; encourage efficient utilisation of healthcare resources; and reduce risk of disease transmission during outbreaks and epidemics. A critical appraisal of the studies was conducted and the findings summarised in Table 2.
Evidence to support the use of ePROMs in routine clinical care Improved patient survival A single-centre RCT by Basch et al. in the United States (US) reported statistically significant associations between the use of the 'STAR' (Symptom Tracking and Reporting) system, a web-based ePROM interface for telemonitoring, and patient survival following chemotherapy for advanced solid tumours. 28 The system incorporates 'patient-adapted questions' from the National Cancer Institute's Common Terminology Criteria for Adverse Events (CTCAE) checklist. 44 It was not clear from the article whether these questions were derived from CTCAE items or the patient-specific PRO-CTCAE. 45 At 1 year after the start of the intervention, patients in the intervention arm, whose symptoms were monitored using STAR, experienced a significant survival advantage over those in the control arm who received routine surveillance. 28 Email reminders were sent to patients randomised to the STAR group and nurses initiated clinical response to alerts generated by patients using the STAR system (see Table 1). The effect of the intervention on survival was greater among participants who had less computer experience, and who were more likely to be older, male, black and less educated. 28 In an assessment of overall survival, conducted after a median follow-up of 7 years, patients in the STAR arm experienced a statistically significant survival advantage of 5 months over patients in the usual care arm. 29 In a multicentre study conducted in France by Denis et al. that further tested the approaches employed by Basch et al., patients with advanced lung cancer following initial treatment were randomly assigned to receive either web-based symptom monitoring via an ePROM system (Hyperion) or standard follow up (scheduled imaging). 35 Interim analysis at 9 months showed that patients in the ePROM group experienced a survival benefit of 7 months. This led to the decision by the independent data monitoring committee to mandate the crossover of patients in the control to the intervention arm. Furthermore, the overall survival rate at 1 year was significantly higher in the ePROM arm than in the control arm (74.9% versus 48.5%). 35 These results were attributed to earlier detection of lung cancer relapse in the ePROM group. In addition, as the ePROM system was reliable in detecting patient relapse, a 49% reduction in CT scans was reported for patients in the ePROM group. Analysis of median overall survival after 2 years of follow up reported that significant survival benefit persisted [22.5 months A clinician has to decide whether further contact is needed. Red: Definite need of contact or the patient asks for a consultation.
• Of 8256 telePRObased contacts from epilepsy outpatients, up to 48% were handled without further contact.
• For sleep apnoea up to 57% of the 1424 telePRObased contacts did not require further contact.
• Completion rates were over 90% at baseline and follow up.
AE, adverse event; AMTRA, ambulatory monitoring of cancer therapy using an interactive application; BPSS, breast cancer patient support system; CI, confidence interval; CT, computerized tomography; CTCAE, common terminology criteria for adverse events; DAS28, disease activity score in 28 joints; ED, emergency department; EORTC QLQ-C30, European organisation for research and treatment of cancer quality of life questionnaire C30; EPR, electronic patient records; ePRO, electronic patient-reported outcome; ER, emergency room; FACT-PWB, functional assessment of cancer therapy scale-general physical well-being subscale; HADS, hospital anxiety and depression scale; HR, hazard ratio; HRQOL, health-related quality of life; IMPROVED, improving management of patient-reported outcomes via electronic data; MSAS, memorial symptom assessment scale; NCI-CTCAE, National Cancer Institute's common terminology criteria for adverse events; PFS, progression free survival (defined as the duration from randomisation to the first radiologic observation of disease progression or to last follow up when the patient is censored); PRO, patient-reported outcome; PROMIS, patient reported outcome measure information system; RA, rheumatoid arthritis; RCT, randomised controlled trial; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; STAR, symptom tracking and reporting; US, United States.  Table 2.
Critical appraisal of included studies. Handa et al. 38 RCT

Articles
It was unclear what constituted usual care and whether the materials provided to the non-ePROM group was also given to ePROM group. Was the assignment of participants to interventions randomised? 3. Were all participants who entered the study accounted for at its conclusion? 4. Were the people analysing outcomes 'blinded'?* (due to the nature of the intervention, patient or investigator blinding was impossible) 5. Were the study groups similar at the start of the randomised controlled trial? 6. Apart from the experimental intervention, did each study group receive the same level of care (that is, were they treated equally)? 7. Were the effects of intervention reported comprehensively? 8. Was the precision of the estimate of the intervention or treatment effect reported? 9. Do the benefits of the experimental intervention outweigh the harms and costs? CASP, critical appraisal skills programme; CI, confidence interval; ePROM, electronic patient-reported outcomes measures; FACT-L, Functional Assessment of Cancer Therapy-Lung; RCT, randomised controlled trial. However, these results must be interpreted with caution. Survival benefits may be age-dependent, as demonstrated by the results of a secondary analysis by Basch et al. 28,29,42 This analysis reported a significantly lower risk of emergency room visits and improved survival among younger patients with solid tumours (median age = 58 years, range 26-69 years) whose symptoms were managed electronically in addition to receiving usual care. 42 These benefits were not observed among older patients who were managed electronically in addition to usual care (median age = 75 years, range 70-91 years). 42 Efficient utilisation of healthcare resources ePROM systems provide medical teams with the opportunity to interact with patients and deliver care efficiently. Changes in patients' clinical status can be monitored remotely and hospital appointments reserved for those that require inperson assessment. Therefore, the use of ePROMs in the routine care of patients with chronic diseases could lead to a more efficient utilisation of limited healthcare resources. In Denmark, the use of a generic ePROM system, Ambuflex, is being used routinely to manage patients across nine chronic conditions (epilepsy, coronary heart disease, narcolepsy, sleep apnoea, prostate cancer, asthma, rheumatoid arthritis, colorectal cancer and kidney failure) to facilitate clinical decisionmaking. 24 Patients complete ePROMs remotely and their data is used to determine whether they require an outpatient hospital appointment. The use of the Ambuflex system led to decreases of 48% and 57% in hospital follow-up visits in patients with epilepsy and sleep apnoea, respectively. 27 The use of this ePROM system across the chronic conditions also led to reductions in (a) the reimbursement of patient transportation costs and (b) the need to destroy excess chemotherapy drugs for the oncology patients (which might have been due to better treatment adherence).
The previously mentioned study that utilised the STAR ePROM system also reported benefits related to resource utilisation. 28 After a year of follow up, patients in the STAR arm had significantly fewer emergency department (ED) visits compared with those who received usual care (34% versus 41%). 28 This effect was more pronounced when the computer-inexperienced subgroups of the two study arms were compared directly (34% versus 56%). The study also reported a significant reduction in hospitalisation among the computer-inexperienced subgroup of the STAR arm (44% versus 63%) but not in the computer-experienced subgroup. 28 Conversely, in the United Kingdom (UK), the RCT of the eRAPID system by Absolom et al. did not find any significant difference in the utilisation of healthcare resources between the intervention and control arms. 33 Specifically, there were no significant differences in chemotherapy delivery, hospital admissions, acute oncology assessments or emergency hotline calls. 33 Improved symptom management An RCT conducted in Sweden evaluated the utility of an interactive 'Interaktor' ePROM app for delivering treatment-related symptom management versus usual care for patients with breast cancer receiving chemotherapy. 37 Patients in the Interaktor group experienced nausea, vomiting, sadness, loss of appetite and constipation less frequently. 37 Overall physical symptom distress was significantly lower and emotional functioning significantly improved in the Interaktor group (see Table 1). 37 De Thurah et al. conducted an RCT comparing disease activity control for rheumatic arthritis using the AmbuFlex system with usual outpatient follow-up care by physicians. Patients with low disease activity who used AmbuFlex experienced similar levels of disease activity control compared with those who received conventional outpatient follow up. 31 Furthermore, aside from the fixed yearly outpatient visits, patients randomised to AmbuFlex required fewer extra visits per year (see Table 1). 31 Based on functional assessment of cancer therapy-general, physical well-being subscale (FACT-PWB) data, there was better symptom control among patients undergoing chemotherapy in the intervention arm of the eRAPID RCT at 6 (p = 0.028) and 12 (p = 0.039) weeks. 33 However, no significant difference was recorded at 18 weeks between the two arms of the study (p = 0.699). 33 journals.sagepub.com/home/taj 15 Mooney et al. evaluated the efficacy of an automated symptom management system in patients commencing chemotherapy in an RCT. 41 All the patients reported their chemotherapy-related symptoms using the system but patients in the symptom care at home (SCH) arm received selfmanagement information and nurse practitioners acted on alerts of poorly controlled symptoms following decision support guidance. 41 At the end of the follow-up period, symptom severity across all symptoms was significantly less in patients randomised to the SCH arm compared with usual care (p < 0.001).
An analysis of PROMs and physician-reported outcomes data, collected as part of an RCT comparing two hypofractionated radiotherapy schedules, was conducted by Rammant et al. 46 The study found poor concordance between patient and physician reported side effects with physicians significantly underrecognising and underreporting patients' symptom burden. 46 Handa et al. conducted an RCT to evaluate the effectiveness of a smartphone-based application to track treatment side effects of patients undergoing chemotherapy. 38 In addition to measuring physical symptoms, the application measured changes in patients' anxiety and depression levels using the hospital anxiety and depression scale (HADS). The study found that medical staff underestimated the severity of 25% of patients' physical symptoms and the most frequently underestimated were muscle/joint pain, fatigue, and nausea. Symptom monitoring in this study did not lead to significant improvements in patients' HADS scores.
The studies by Rammant and Handa confirm previous findings that medical teams underestimate symptoms compared with patient reports. 47,48 They also highlight the need for a mechanism, such as ePROMs, by which patients can provide assessments of their health status to complement clinician assessments.
In an RCT conducted by Kroenke et al., patients who screened positive for at least one SPADE (sleep, pain, anxiety, depression and low energy/ fatigue) symptom were recruited (their underlying chronic conditions were unspecified). 40 They completed the five matching domains of the patient reported outcome measure information system (PROMIS) profile-29 electronically and were subsequently randomised to a feedback group in which their clinician only received a visual display of their symptom scores or a control group in which the scores were not provided to clinicians. 40 At the end of the 3-month follow-up period, while both groups had moderate symptom improvement, there was a non-significant trend favouring the feedback compared with the control group (see Table 1). 40 There is a possibility that participants in the RCTs by Handa and Kroenke did not experience a significant improvement in their symptoms because there were no specific interventions to address issues highlighted by the ePROM systems. 38,40,49 While this might be acceptable for an RCT, depending on the study design and aims, collecting and analysing PROM data in a routine setting without acting on the information by providing treatment/support for problems identified could be a waste of resources. 49 Kroenke et al. concluded that providing clinicians ePROM data without additional systems support or incentives is insufficient. 40 Medical teams need to carefully consider how ePROMs data will be translated into actionable plans for patient management when developing ePROM systems.

Improved treatment adherence and monitoring
A multi-centre, non-randomised prospective cohort analysis performed in Belgium compared ePROMs collection using a web-based system (AMTRA) with usual care among patients with solid tumours undergoing chemotherapy. 43 Patients in the intervention arm used an application to provide daily reports of the severity of their symptoms. 43 They were allowed to provide additional reports if required. Self-care information was provided for mild or moderate symptoms, and alerts were sent to the medical staff in cases of severe adverse events. Patients experienced a statistically significant reduction of mean severity grade for nausea, constipation, loss of appetite, fatigue and dyspnea over the follow-up period. 43 The AMTRA app uniquely incorporated a treatment adherence aid that required patients to log their medication intake. An automatic reminder was sent to the provider if an entry was not logged and followed up within 24 h if there was still no response. There was a median treatment adherence rate of 98.7% in the 44 patients receiving oral chemotherapy. 43 The study's success may also be attributed to the training that was provided to patients and medical staff prior to study commencement. Medical staff received training on the system and the pathways for responding to patientgenerated alerts. 43 Patients had home visits during which they were shown how to use the application by trainers who emphasised the importance of reporting symptoms and adhering to treatment.

Reduced risk of disease transmission during disease outbreaks
The COVID-19 pandemic has emphasised the need and relevance of ePROMs in clinical trials and routine practice. 50 There is now a renewed interest in all forms of telemedicine and the effect will probably remain even after the threat of COVID-19 has subsided.
When the COVID-19 epidemic began, it quickly became apparent that providing medical care outside hospital settings reduced the risk of exposure to, and transmission of, the SARS-CoV-2 virus for patients and clinicians. 50 Kricke et al. recently reported early findings from the implementation of an outpatient COVID-19 monitoring program in the US. Over 6000 patients with presumed COVID-19 infection completed an electronic daily symptom questionnaire from home and were stratified based on symptom severity. While the majority of the patients did not require hospitalisation, remote monitoring led to emergency department referrals for an average of nine patients per day. 39 With the growing incidence of long COVID, the use of ePROMs for symptom monitoring could facilitate the identification of life-threatening complications that may develop later in patients.

Challenges with the use of ePROMs
While the studies included in this review demonstrate potential benefits of using ePROMs in patient care, there are a number of challenges that need to be adequately addressed to ensure seamless implementation and integration of ePROMs into a health system. 51

Maintaining high ePROM completion rates
The use of ePROMs could precipitate patients' experiences of survey fatigue, which may lead to a steady decline in survey completion over time. 52 Other individual and system level factors which may be associated with reduced propensity to complete surveys or to higher survey fatigue, include: older age, disease severity, the presence of comorbidities, 11 questionnaire length and item relevance, and perceptions of response burden. 53 Participant fatigue may be minimised with innovative PROM assessment methods. For instance, computerised adaptive testing (CAT), an algorithm-based method, tailors ePROMs to the individual by automatically selecting and sequentially administering the most relevant items (questions) from an item bank based on a respondent's prior responses. [54][55][56] With the expansion of ePROMs, the expansion of CATs and other methods to improve survey design and reduce survey burden should be considered. 57,58 The AmbuFlex system attributed its high completion rates to the use of a mixed-mode (paper and web) method of data collection. 27 A recent publication by Niels Hjollund reflected on the 15-year use of the AmbuFlex system for the follow-up of patients with chronic diseases. He noted that although a mixed-mode method of collection of PROM data was initially implemented to maximise response rates (66.5% of responses were paper-based in 2005), there has been a gradual preference for an electronic option (only 4.3% were paper-based in 2019). 59 Despite a decline from an initial baseline completion rate of 94%, Taarnhoj et al. still reported relatively high completion rates of more than 70% over six cycles of chemo-or immunotherapy. The rates only declined after patients had received the sixth cycle of treatment. 60 In the study by Kricke et al., 20% of patients monitored for COVID-19 symptoms did not fill out the daily questionnaire, particularly those with mild symptoms. 39 Missing data is also a common problem with patientreported data. Identifying causes of missing entries could improve the clinical utility of ePROMs and highlight opportunities to improve ePROM completion. 61 The level of health literacy among patients may significantly influence their decisions to engage with ePROM interventions. 62 To encourage compliance, efforts need to be made to ensure that patients understand the importance and potential benefits of providing ePROM data in relation to their clinical management and health outcomes.

Cost and cost-effectiveness
The issues of cost and cost-effectiveness could significantly influence the decision by health care providers to commission the development and implementation of ePROM systems. For policymakers, the cost and cost-effectiveness of ePROM interventions in comparison with existing follow up care may determine whether crucial governmental and/or institutional support in terms of legislation or finance is secured.
Data from a recent trial compared the cost-effectiveness of usual care, tele-rehabilitation, and tele-rehabilitation plus pharmacological pain management for patients with late-stage cancer (details about cancer types not provided). 63 At the $100,000.00 willingness-to-pay threshold, the tele-rehabilitation model was the most costeffective strategy in 95% of simulations. 63 The authors attributed the cost savings to the fact that patients in the tele-rehabilitation arms were less likely to require intensive care unit admission. 63 Cost-effectiveness analysis was pre-specified as a secondary outcome in an RCT by Denis et al. 35  An awareness of the tensions and challenges experienced by HCPs with ePROMs and their engagement and involvement in ePROM system development, implementation and integration is essential to overcome these barriers. 11,[66][67][68]70,71 Clear guidelines or actionable plans are essential to enable clinicians respond confidently and effectively to ePROM data. 38,40,41,49 Integration of ePROMs into existing health systems The successful integration of ePROM into existing health systems and workflows has the potential to transform clinical practice. 72 This process may be complex, resource-intensive, requiring an iterative approach and stakeholder involvement. 51,70,[72][73][74][75] Appropriate training of HCPs to handle and respond to ePROM data needs to be provided in order to facilitate integration with existing clinical workflows. 43,66,76 LeRouge et al. have recently published a comprehensive toolkit to facilitate the process of implementation and integration of ePROMs with existing health systems. 77 This toolkit comprises of guidelines that focus on the various aspects of the process including ePROM integration approaches, workflow designing, leveraging of health ICT, and display of data. 77

Ongoing ePROM initiatives
There is growing evidence that the thoughtful incorporation of ePROMs into health systems could have meaningful and positive impacts on clinical outcomes. However, most of this evidence to support their use originates from the field of oncology. Thus, there is a need to demonstrate its potential impact in other chronic conditions. Initial work is underway to explore the use of ePROMs in the routine care of patients with CKD. The RePROM study is currently being conducted by researchers and clinicians based at the Centre for Patient-Reported Outcomes Research at the University of Birmingham and the Queen Elizabeth Hospital Birmingham within the UK National Health Service University Hospitals Birmingham Foundation Trust (UHBFT). 78 The study will determine the feasibility of undertaking a full-scale RCT of ePROMs in the symptom monitoring of patients living with advanced CKD. 78 A pilot study also involving patients with CKD at the University Health Network, Toronto, Canada, will test the feasibility of implementing an ePROM toolkit to assess physical and emotional symptoms, using the PROMIS CATs. This is linked to a symptom management, self-management support resource hub. 79,80 The Evaluation of Routinely Measured Patientreported Outcomes in Haemodialysis Care (EMPATHY) RCT, which aims to determine the effects of routine measurement of ePROMs on the experiences of patients undergoing in-centre haemodialysis in Alberta and Ontario, Canada, is currently underway. 81 The use of ePROMs in the management of patients receiving home dialysis is also being trialled in the ePRO Kidney study. 82

Conclusions
There is increasing evidence that the use of ePROMs could have significant impacts on outcomes valued by patients, healthcare providers and researchers. While our paper focusses primarily on quantifiable effects of ePROMs on traditional clinical outcomes, this does not mean that other outcomes are less important. 83 As most of the evidence we found was derived from oncology, there is an urgent need for similar work in other subspecialties that care for medically complex patients. ePROM systems that demonstrate maximum clinical utility and minimum patient burden need to be seamlessly integrated into a learning health system. Whilst the development and implementation of these systems may be initially costly and resource-intensive, patient preferences and existing evidence to support their implementation suggests the need for continued research prioritisation in this area. In addition, there is a need for case studies to demonstrate best practices for achieving and maintaining patient and HCP engagement. Finally, further research is required to demonstrate the long-term cost-effectiveness of using ePROMs in routine clinical practice. Significant work remains, but the implementation of ePROMs provides a valuable opportunity to transform the quality and delivery of care for medically complex patients.

Conflict of interest statement
OLA has received personal fees from Gilead Sciences Ltd, Merck and GSK outside the submitted work. All other authors declare that they have no competing interests. generic integrated PRO system, WestChronic. Interact J Med Res 2014; 3: e5.